[JAVA-2294] GridFSDownloadStream - exception when destination buffer size != file chunk size Created: 01/Sep/16  Updated: 19/Oct/16  Resolved: 09/Sep/16

Status: Closed
Project: Java Driver
Component/s: API
Affects Version/s: 3.3.0
Fix Version/s: 3.4.0

Type: Bug Priority: Major - P3
Reporter: Sean Fitts Assignee: Ross Lawley
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

When using GridFSDownloadStream to download a file with multiple chunks, if you use a destination buffer whose size is not exactly equal to the file's chunk size then you will get the following exception:

com.mongodb.MongoGridFSException: Could not find file chunk for files_id: BsonObjectId{value=57c87e6afc76010ce8f43887} at chunk index 2.
 
	at com.mongodb.async.client.gridfs.GridFSDownloadStreamImpl.chunkNotFound(GridFSDownloadStreamImpl.java:235)
	at com.mongodb.async.client.gridfs.GridFSDownloadStreamImpl.access$600(GridFSDownloadStreamImpl.java:38)
	at com.mongodb.async.client.gridfs.GridFSDownloadStreamImpl$5.onResult(GridFSDownloadStreamImpl.java:168)
	at com.mongodb.async.client.gridfs.GridFSDownloadStreamImpl$5.onResult(GridFSDownloadStreamImpl.java:162)
	at com.mongodb.operation.AsyncQueryBatchCursor.next(AsyncQueryBatchCursor.java:109)
	at com.mongodb.async.client.gridfs.GridFSDownloadStreamImpl.checkAndFetchResults(GridFSDownloadStreamImpl.java:162)
	at com.mongodb.async.client.gridfs.GridFSDownloadStreamImpl.read(GridFSDownloadStreamImpl.java:132)

I have created a GIST for a JUnit test which demonstrates the issue – https://gist.github.com/sfitts/6ac8fc6efa19ef86600ae5354682c5ca

The issue seems to be that the code decides to do a read-ahead into a chunk that doesn't exist (it is one off the edge) and so doesn't get to use the data already sitting in "buffer" to fulfill the read.



 Comments   
Comment by Githook User [ 09/Sep/16 ]

Author:

{u'username': u'rozza', u'name': u'Ross Lawley', u'email': u'ross.lawley@gmail.com'}

Message: Fix Async GridFS Download to Stream small buffer bug

When using a smaller than chunksize buffer, need to check the internal buffer has been read before trying to get the next chunk.

JAVA-2294
Branch: 3.3.x
https://github.com/mongodb/mongo-java-driver/commit/4cd3e84e4a547a8202313f04b46930218f5f15b8

Comment by Githook User [ 09/Sep/16 ]

Author:

{u'username': u'rozza', u'name': u'Ross Lawley', u'email': u'ross.lawley@gmail.com'}

Message: Fix Async GridFS Download to Stream small buffer bug

When using a smaller than chunksize buffer, need to check the internal buffer has been read before trying to get the next chunk.

JAVA-2294
Branch: master
https://github.com/mongodb/mongo-java-driver/commit/c39037ec809f8c4d7bdebbfa9a8a86d5bdd4e4e5

Comment by Sean Fitts [ 05/Sep/16 ]

No problem – thanks for the quick fix.

Comment by Ross Lawley [ 05/Sep/16 ]

Thanks smfitts@gmail.com,

For the ticket, the gist made it easy to reproduce the bug. Unfortunately, it was missed in the test case for this exactly this issue - in the test case the chunk size wasn't a multiple of the chunk size, so eventually crossed chunk boundaries as expected.

Comment by Ross Lawley [ 05/Sep/16 ]

PR: https://github.com/rozza/mongo-java-driver/pull/159

Comment by Sean Fitts [ 03/Sep/16 ]

Note that a value greater than chunk size works (as long as it is a multiple of chunk size), but lower values do not. If you ever get to the point where there is data left to read out of the internal field "buffer", but no more chunks remain then the exception above is thrown.

Generated at Thu Feb 08 08:56:51 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.