Details
Bug
Closed
Critical - P2
Description
Interleaving of concurrent appended writes to a GridStore file result in an uncaught throw in the driver resulting from file corruption and/or data loss due to duplicate chunk numbering when the writes cause one or more chunks to be created or added to.
Attached script synthetically reproduces the issue in a single process thread. The same issue may occur with multiple processes accessing the same GridFS collection simultaneously.
Activity
Sent pull request, thanks.
Disabled w+ mode until there can be a longterm real solution to this.
In node mongodb native drive the "w+" mode is still authorized (it doesn't raise any error), although it's not documented.
https://github.com/mongodb/node-mongodb-native/blob/94045c0b658edf8c7c87d55cee79083494a9762a/lib/gridfs/grid_store.js#L166
- GridStore only supports w+ for metadata updates, no appending to file as it's not thread safe and can cause corruption of the data
+ seek will fail if attempt to use with w or w+
+ write will fail if attempted with w+ or r
+ w+ only works for updating metadata on a file
Okay.
This could be explained in the documentation, because today we have conflicting information:
at the beginning of GridStore doc: mode can be "r" or "w", and multiple methods docs talk about 'mode "w" or "w+"'.
A cursory review of the gridstore code reveals no mechanism for locking the .files collection entry for a file while it undergoes writes to its chunk structure. Since under the GridFS data model, writes to the .chunks and .files collections are independent, no protection for concurrent access is provided by the mongodb core. Some type of atomicity guarantee for file writes perhaps using a findAndModify() based semaphore on the .files document for a file is necessary to safeguard against dataloss under attempted concurrent write access.