[GODRIVER-1548] Access filename and metadata in gridfs DownloadStream Created: 26/Mar/20 Updated: 28/Oct/23 Resolved: 02/Apr/20 |
|
| Status: | Closed |
| Project: | Go Driver |
| Component/s: | GridFS |
| Affects Version/s: | None |
| Fix Version/s: | 1.4.0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Ed Pelc | Assignee: | Divjot Arora (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Description |
|
The api for gridfs does not give you access to the filename or metadata of a file when downloading it even though internally it fetches the file document. I propose adding fields or methods to access these on `DownloadStream` and internally just linking up to the file document which is already fetched. This would prevent user libraries from having to implement this and also from having to redundantly lookup the file document(ie read from the db again). Our app uses file ids to lookup files but we use the name and metadata(contenttype stored here) to provide a more friendly url to the user. While upgrading from globalsign/mgo I found that the gridfs implementation here doesn't give you any way to read file names or metadata easily. Right now you have to hack around it by fetching the file again and decoding it yourself. |
| Comments |
| Comment by Githook User [ 02/Apr/20 ] |
|
Author: {'name': 'Divjot Arora', 'email': 'divjot.arora@10gen.com', 'username': 'divjotarora'}Message: |
| Comment by Divjot Arora (Inactive) [ 02/Apr/20 ] |
| Comment by Ed Pelc [ 31/Mar/20 ] |
|
Awesome! Thanks for this!
This really simplifies upgrading gridfs. Will be nice to tear out my workaround. |
| Comment by Divjot Arora (Inactive) [ 31/Mar/20 ] |
|
Now that we've reached an agreement about the best way to proceed with this, I'm going to move it from "Investigating" to "Scheduled". We will complete the work and release this in the 1.4.0 driver release. Thank you for the quick responses and ideas! – Divjot |
| Comment by Divjot Arora (Inactive) [ 31/Mar/20 ] |
|
Yeah, interface{} isn't correct because that'd be unmarshalled as a bson.D (the driver defaults to bson.D, whereas mgo defaults to bson.M because bson.D is an ordered type and therefore inherently safer). Because the metadata field is guaranteed to be a document, we can keep it as a bson.Raw, which is the raw bytes comprising a BSON document and a user can safely call bson.Unmarshal. I like this because this limits the performance overhead for someone who just needs to access one field, but also allows you to easily unmarshal the whole thing into a custom Go type if needed. |
| Comment by Ed Pelc [ 31/Mar/20 ] |
|
That would work you just have to be sure the underlying data is not auto unmarsahlled into something generic like a bson.M before it gets to the user code. That is much cleaner too! Good idea. |
| Comment by Divjot Arora (Inactive) [ 31/Mar/20 ] |
|
If the metadata was kept as interface{}, would it be the same to have the user call bson.Unmarshal(file.Metadata, &result)? |
| Comment by Ed Pelc [ 31/Mar/20 ] |
|
I think that would work well! Is there anyway we could support unmarshalling the metadata into a go struct or other custom type instead of just getting back a `bson.M` or similar? This would make it easy to have a full object/nested fields or any other custom data and make the metadata have full power vs having to again do a separate file lookup using regular `FindOne().Decode()`.
Almost like you'd make an `UnmarshalMetadata(&userSuppliedOutVar)` method on the `GridFSFile` type instead of the java version which returns a Document(similar to bson.M?) http://mongodb.github.io/mongo-java-driver/4.0/apidocs/mongodb-driver-core/com/mongodb/client/gridfs/model/GridFSFile.html?is-external=true#getMetadata() I know the current implementation basically pulls out the name and id field without unmarshalling the entire document so I'd assume it'd be simple to delay the metadata unmarshalling. |
| Comment by Divjot Arora (Inactive) [ 31/Mar/20 ] |
|
epelc@greatcloak.com Thank you for the detailed feature request! Instead of multiple methods on DownloadStream to access each piece of file information, I propose adding a new gridfs.File type and a DownloadStream.GetFile method to return the file being downloaded. These would be similar to the GridFSFile type and GridFSDownloadStream.getGridFSFile() method in the Java driver (see http://mongodb.github.io/mongo-java-driver/4.0/apidocs/mongodb-driver-core/com/mongodb/client/gridfs/model/GridFSFile.html?is-external=true and http://mongodb.github.io/mongo-java-driver/4.0/apidocs/mongodb-driver-sync/com/mongodb/client/gridfs/GridFSDownloadStream.html#getGridFSFile()). Does this sound like a reasonable proposal to you? |