[GODRIVER-724] Downloading files from gridfs does not match uploaded data Created: 19/Dec/18  Updated: 28/Oct/23  Resolved: 08/Jan/19

Status: Closed
Project: Go Driver
Component/s: GridFS
Affects Version/s: 0.1.0
Fix Version/s: 0.2.0

Type: Bug Priority: Critical - P2
Reporter: Ronnie van der Veen Assignee: Devin Hilly (Inactive)
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

mongodb v3.6.9
ubuntu1604


Attachments: File main.go    

 Description   

See attached main.go file

  • it reads a file from /tmp/infile and uploads it to GridFS
  • downloads it from GridFS and writes it to /tmp/outfile

I'd expect the input and output files to be the same size and have the same content, but for file sizes smaller than the default chunksize the downloaded file seems to be "padded" so it ends up being 256kb.

For big files (say 12MB) the resulting outfile does not match the infile.  The size is often smaller, and big chunks of data just seem to be nulls instead of the actual file data.

 

I also had issues uploading a 16MB video file, then the UploadFromStream returns the following error:

2018/12/19 16:21:37 Uploading to mongo2018/12/19 16:21:37 Uploading to mongopanic: runtime error: index out of range
goroutine 1 [running]:github.com/mongodb/mongo-go-driver/mongo/gridfs.(*UploadStream).uploadChunks(0xc0000bc2d0, 0x985940, 0xc0001680c0, 0x985940, 0xc0001680c0) /pkg/mod/github.com/mongodb/mongo-go-driver@v0.1.0/mongo/gridfs/upload_stream.go:163 +0x7bfgithub.com/mongodb/mongo-go-driver/mongo/gridfs.(*UploadStream).Close(0xc0000bc2d0, 0x0, 0x0) /pkg/mod/github.com/mongodb/mongo-go-driver@v0.1.0/mongo/gridfs/upload_stream.go:70 +0xfdgithub.com/mongodb/mongo-go-driver/mongo/gridfs.(*Bucket).UploadFromStreamWithID(0xc0001be000, 0x2bb9ac88f1531a5c, 0xc0c125834f, 0x85a272, 0x9, 0x9812c0, 0xc0001ae000, 0x0, 0x0, 0x0, ...) /pkg/mod/github.com/mongodb/mongo-go-driver@v0.1.0/mongo/gridfs/bucket.go:174 +0x1aegithub.com/mongodb/mongo-go-driver/mongo/gridfs.(*Bucket).UploadFromStream(0xc0001be000, 0x85a272, 0x9, 0x9812c0, 0xc0001ae000, 0x0, 0x0, 0x0, 0x0, 0xc000000000, ...) /pkg/mod/github.com/mongodb/mongo-go-driver@v0.1.0/mongo/gridfs/bucket.go:138 +0xd1main.Set(0xc0001a2060, 0x85a272, 0x9, 0x9812c0, 0xc0001ae000, 0x0, 0xc0001a2060) /src/bitbucket.org/mybeat/sortd_go/cmd/testmongo/main.go:70 +0x17bmain.main() /src/bitbucket.org/mybeat/sortd_go/cmd/testmongo/main.go:40 +0x33e

 

 

I ran the same test using the mgo driver and there it worked as expected.

 



 Comments   
Comment by Githook User [ 08/Jan/19 ]

Author:

{'username': 'dmhilly', 'email': 'devin.hilly@mongodb.com', 'name': 'Devin Hilly'}

Message: Fixed bugs in gridfs implementation

GODRIVER-724

Change-Id: I5afcd5b7dc810f1696a7ba1f258f3a80c4d0284c
Branch: master
https://github.com/mongodb/mongo-go-driver/commit/2c8e308599d5b93871a9bfa75405148aea0d9257

Comment by Ronnie van der Veen [ 19/Dec/18 ]

What I forgot to mention with the first part of the report is that when I download the file from mongodb using another tool (NoSQLBooster for MongoDB) the file is correct.  So the issue seems to be with downloading the file, not during the upload.

Generated at Thu Feb 08 08:34:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.