[CSHARP-3204] Support setting GridFS metadata at the end of uploading to a stream Created: 03/Sep/20  Updated: 22/Jun/22

Status: Blocked
Project: C# Driver
Component/s: GridFS
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Minor - P4
Reporter: Thomas Tutko Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Backwards Compatibility: Fully Compatible

 Description   

Currently, when uploading to a stream using the sample code provided such as:

using (var stream = bucket.OpenUploadStream("filename", options))
{
    var id = stream.Id; // the unique Id of the file being uploaded
    // write the contents of the file to stream
    stream.Close();
}

The metadata document needs to be known ahead of time. However, there may be scenarios where it could be useful to set/modify the metadata after the upload of the file itself is complete. The primary example is storing the hash of a file being created on the fly where the value is not known until the file is done being written. Another example might be to store the duration of the time it took to upload the file.

I fully recognize that nothing stops us from updating the metadata after the fact but this results in another round trip to the database that is not needed since the current implementation doesn't create the files document until after the upload already. My suggestion would be an overload to OpenUploadStream and OpenUploadStreamAsync that would allow for a callback to update the metadata:

public GridFSUploadStream<TFileId> OpenUploadStream(TFileId id, string filename, GridFSUploadOptions options = null, Func<BsonDocument, BsonDocument> meatdataCallback = null, CancellationToken cancellationToken = default(CancellationToken))

This callback lambda would get the original metadata BsonDocument passed to it and would return the final metadata BsonDocument to be stored. This would of course have to be carried through to GridFsForwardOnlyStream and called (if provided) during CreateFilesCollectionDocument() (which already doesn't get called until the end anyhow so I don't think this would be too difficult).



 Comments   
Comment by Robert Stam [ 14/Sep/20 ]

Your proposal does not "violate" the spec... it just isn't "in" the spec.

The spec includes a definition of the API (i.e. methods like OpenUploadStream) available to upload files. If we are going to add a new way to upload files (specifying the metadata at the end of the upload instead of at the beginning) that needs to be added to the spec.

Comment by Thomas Tutko [ 14/Sep/20 ]

@Robert Stam can you please point me to the part of the specification that would be required to change to allow for this? I just looked over the spec at GridFS Spec and I don't see how this would violate the spec or change the spec in any way. The files collection document is already, according to the spec, the last piece to be written.

Comment by Robert Stam [ 14/Sep/20 ]

A change like this would require a change to the GridFS spec which would then be implemented in all the drivers.

Generated at Wed Feb 07 21:44:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.