[GODRIVER-2142] unexpected EOF - large binary file (>1mb and <10mb) Created: 01/Sep/21 Updated: 27/Oct/23 Resolved: 27/Sep/21 |
|
| Status: | Closed |
| Project: | Go Driver |
| Component/s: | BSON |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | punit naran | Assignee: | Benji Rewis (Inactive) |
| Resolution: | Gone away | Votes: | 0 |
| Labels: | Bug | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Description |
|
Replicate this error - Create a zip file containing multiple files of different sizes which can come up to 10mb. Store that file as in a document something like: name: "hello" - string lifetime: 8943 - int config_file: Binary('...', 0) - zip is stored as binary format
After uploading this and then using golang to marshal it into a struct: by using a base64 decoder and then using ioutil.ReadAll this successfully decodes and returns a list of bytes (the zip file represented in bytes).
Issue: Running this multiple times I've noticed it would fetch and decode successfully multiple times and rarely it would decode and report "unexpected EOF" error. I've run my own mongo instances used a json file that adds the document to the collection and then used golang to read the document.
The json file is not changing it remains the same. Most of the time it works fine, and sometimes it fails. |
| Comments |
| Comment by PM Bot [ 27/Sep/21 ] | |||||||||
|
There hasn't been any recent activity on this ticket, so we're resolving it. Thanks for reaching out! Please feel free to comment on this if you're able to provide more information. | |||||||||
| Comment by Benji Rewis (Inactive) [ 10/Sep/21 ] | |||||||||
|
Great thank you, pnaran@cryptoquantique.com! I’m still a little confused about your use-case. I’ve attached some Go code that is a cleaned up version of the sample code you’ve provided. I’m assuming your code is similar to the one I’ve attached, and you’re getting an occasional unexpected EOF from the ioutil.ReadAll call on L36. My main question is: what does your data look like in your collection? From what I can tell, you seem to be storing your file in a collection like this:
Is that correct? You have three cases in your provided code. One where data_file is a map[string]map[string]string (as is the case above), one where data_file is a plain string, and one where neither is true (in which case you error). The first case makes sense to me, and calling DecodeString on [base 64 encoding of zip file] would certainly return the zip file represented in bytes. The second case, where data_file is a plain string, is confusing to me. What is the value of data_file here? It looks like you’re treating the string value as a list of bytes and encoding it to a base64 string. But, then you call ioutil.ReadAll on that string, which is not possible, as the argument to ReadAll must implement the io.Reader interface. Are you sure you’re calling ioutil.ReadAll(sEnc)? As an aside, if you’re storing larger, whole files with MongoDB, I would recommend using the GridFS feature. | |||||||||
| Comment by punit naran [ 09/Sep/21 ] | |||||||||
| |||||||||
| Comment by Benji Rewis (Inactive) [ 08/Sep/21 ] | |||||||||
|
pnaran@cryptoquantique.com apologies for the delay, and thanks for your bug report! We're investigating now. Some preliminary questions:
|