Uploaded image for project: 'Node Driver'
  1. Node Driver
  2. NODE-157

Concurrent GridStore file append operations result in file corruption / data loss, uncaught throw

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical - P2
    • Resolution: Fixed
    • Affects Version/s: 1.3, 1.4, 3.0
    • Fix Version/s: 2.0
    • Environment:
      All
    • Operating System:
      ALL
    • # Replies:
      25
    • Last comment by Customer:
      true

      Description

      Interleaving of concurrent appended writes to a GridStore file result in an uncaught throw in the driver resulting from file corruption and/or data loss due to duplicate chunk numbering when the writes cause one or more chunks to be created or added to.

      Attached script synthetically reproduces the issue in a single process thread. The same issue may occur with multiple processes accessing the same GridFS collection simultaneously.

      1. concurrent_append_failure.js
        2 kB
        Vaughn Iverson
      2. gridstore.js
        52 kB
        Cristian Ianto

        Activity

        Hide
        iantocristian Cristian Ianto added a comment - - edited

        Sent pull request, thanks.

        Show
        iantocristian Cristian Ianto added a comment - - edited Sent pull request, thanks.
        Hide
        christkv Christian Amor Kvalheim added a comment -

        Disabled w+ mode until there can be a longterm real solution to this.

        Show
        christkv Christian Amor Kvalheim added a comment - Disabled w+ mode until there can be a longterm real solution to this.
        Hide
        trsystran Thomas Riccardi added a comment -

        In node mongodb native drive the "w+" mode is still authorized (it doesn't raise any error), although it's not documented.
        https://github.com/mongodb/node-mongodb-native/blob/94045c0b658edf8c7c87d55cee79083494a9762a/lib/gridfs/grid_store.js#L166

        Show
        trsystran Thomas Riccardi added a comment - In node mongodb native drive the "w+" mode is still authorized (it doesn't raise any error), although it's not documented. https://github.com/mongodb/node-mongodb-native/blob/94045c0b658edf8c7c87d55cee79083494a9762a/lib/gridfs/grid_store.js#L166
        Hide
        christkv Christian Amor Kvalheim added a comment -
        • GridStore only supports w+ for metadata updates, no appending to file as it's not thread safe and can cause corruption of the data
          + seek will fail if attempt to use with w or w+
          + write will fail if attempted with w+ or r
          + w+ only works for updating metadata on a file
        Show
        christkv Christian Amor Kvalheim added a comment - GridStore only supports w+ for metadata updates, no appending to file as it's not thread safe and can cause corruption of the data + seek will fail if attempt to use with w or w+ + write will fail if attempted with w+ or r + w+ only works for updating metadata on a file
        Hide
        trsystran Thomas Riccardi added a comment -

        Okay.
        This could be explained in the documentation, because today we have conflicting information:
        at the beginning of GridStore doc: mode can be "r" or "w", and multiple methods docs talk about 'mode "w" or "w+"'.

        Show
        trsystran Thomas Riccardi added a comment - Okay. This could be explained in the documentation, because today we have conflicting information: at the beginning of GridStore doc: mode can be "r" or "w", and multiple methods docs talk about 'mode "w" or "w+"'.

          People

          • Votes:
            3 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Days since reply:
              4 weeks, 2 days ago
              Date of 1st Reply: