[CDRIVER-545] libbson does not handle large arrays well Created: 26/Feb/15  Updated: 03/May/17  Resolved: 21/May/15

Status: Closed
Project: C Driver
Component/s: None
Affects Version/s: 1.0.2
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Paul Childs Assignee: Unassigned
Resolution: Done Votes: 0
Labels: driver
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Both Ubuntu & Amazon Linux


Backwards Compatibility: Fully Compatible

 Description   

I have an array in Mongo with a large number of values (e.g. I'm storing some json WebGL models where there are an array of vertices, very big and due to the specification no changes aloud there).
The mongo driver works well with uploading but when downloading via libbson my arrays get truncated.
E.g. I have a document with an array of size 1250 amongst other significant stuff. When I run through it with a bson_iter I only count 1185 elements.
The problem is much much worse when I try to run bson_to_json as the array gets truncated down to ~214 and whats worse there is a trailing comma at the end of the list (I think it collapsed an empty element due to the first truncation) and it won't parse.
I am well well under the size limit of the document (4kb BSON) and expect to be able to deal with arrays of size O(100,000).



 Comments   
Comment by Mira Carey [ 26/Feb/15 ]

That absolutely shouldn't be happening.

Can you provide a little more information on how exactly you're using libmongoc and libbson to pull bson over the network? I suspect that what you're actually seeing is corruption, rather than libbson being broken with arrays. You can check that out by:

  • calling bson_validate() on the bson document that appears to have truncated arrays. If that returns false, you'll know you've got corruption.
  • writing the bson bytes to a file and attaching it to this ticket. You can access the raw bytes with bson_get_data() and the length with bson->len. Once I can see what you're actually working with I can make a fix if one's needed.

Regards,
Jason

Generated at Wed Feb 07 21:09:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.