Uploaded image for project: 'Node.js Driver'
  1. Node.js Driver
  2. NODE-372

GridFS: what do do in case of write error?

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 2.0.16
    • Component/s: None
    • Labels:

      Writing a nodejs Stream2 Writable interface to GridFS raised some questions, particularly on error handling:

      What to do when writing to a GridFS file failed (write() error, or close() error)?

      • If we do nothing, then Chunk documents will leak, because the File document referencing them may not have been written.
      • If we remove the written Chunks, the real goal would ideally be to revert to the previous state for the file:
        • if new file: just remove the chunks
        • if overwrite old file: we should remove the new chunks, but keep the old ones: it seems the driver 2.0 first removes the old chunks during open in "w" mode, without touching the File document, so it's not feasable, (and it also creates a timeframe where the File document exists but the GridFS file is corrupted (no chunks, read will fail))

      Also, what to do when the GridFS file write is cancelled? (it happens if the source of the data written to the GridFS file fails itself, for example streaming a http request to GridFS: the tcp connection can fail in the middle of the write)

      • If we do nothing, we also have Chunk documents leak.
      • If we complete the write (GridStore.close()), then we write an incomplete file to GridFS.
      • If we rollback the write: we cannot do this easily, see previous point.

      What are the official recommentations around those issues?

      Ideally I think we should rollback on error, like AWS S3 proposes: write are atomic.
      But GridFS is probably not well designed for that.
      Maybe we should complete the write, but with an error flag in `metadata`, and the user could remove or not the new version (the old version in case of overwrite is already lost anyway).

      Related discussion:
      https://github.com/aheckmann/gridfs-stream/pull/67
      https://jira.mongodb.org/browse/NODE-343

            Assignee:
            christkv Christian Amor Kvalheim
            Reporter:
            trsystran Thomas Riccardi
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: