[SERVER-14123] some operations can create BSON object larger than the 16MB limit Created: 31/May/14 Updated: 09/Mar/21 Resolved: 09/Jun/14 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Querying |
| Affects Version/s: | None |
| Fix Version/s: | 2.6.4, 2.7.4 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Kevin Locke | Assignee: | David Storch |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||
| Backport Completed: | |||||||||||||||||||||||||
| Steps To Reproduce: | Install 2.6.1 and run the following script:
|
||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Linked BF Score: | 0 | ||||||||||||||||||||||||
| Description |
|
Issue Status as of Jul 22, 2014 ISSUE SUMMARY
In these scenarios, MongoDB fails to check that the BSON format doesn't grow beyond the 16MB limit. This can be the case for large query predicates, such as queries that contain an $in with thousands of elements. USER IMPACT WORKAROUNDS AFFECTED VERSIONS FIX VERSION RESOLUTION DETAILS Original descriptionRunning a query for a large number of _ids results in error code 10334 due to a BSON object larger than the maximum size. The error is not in the query object itself (which is ~4MB in the example) but internally from what appears to be the query planner. Such a query generates the following message in the log:
The issue is partially that a valid query object significantly below the 16MB limit is rejected, but to me the issue is primarily that this appears to be an arbitrary query limit which can't be determined a priori and is likely dependent on implementation details that may change between versions. Is there a way to determine if such queries will run (for particular server versions or for all versions), or Is the only solution for large queries to try them and cut them in half (whatever that means for queries on multiple keys) whenever they don't work? Updating the documentation to make a note of this limitation could also be useful to others in a similar situation in the future. Thanks for considering, |
| Comments |
| Comment by Githook User [ 25/Jul/14 ] |
|
Author: {u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}Message: explain_large.js spawns 18 shards and runs a query that allocates (cherry picked from commit 4dd1c9e70cc4a17d233c32180933da24b4c57ef5) |
| Comment by Githook User [ 25/Jul/14 ] |
|
Author: {u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}Message: explain_large.js spawns 18 shards and runs a query that allocates |
| Comment by Githook User [ 24/Jul/14 ] |
|
Author: {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}Message: (cherry picked from commit ffc53f46956eb52a86efd51620dede98e1b48444) |
| Comment by Githook User [ 24/Jul/14 ] |
|
Author: {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}Message: |
| Comment by Githook User [ 23/Jul/14 ] |
|
Author: {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}Message: (cherry picked from commit 906cfb13a337fdbdf1b94b773390c4230b059fec) |
| Comment by Daniel Pasette (Inactive) [ 23/Jul/14 ] |
|
Dave will backport the current fix to 2.6 as well as work on a separate patch for mongos to ensure a similar thing can't happen there. |
| Comment by Kevin Locke [ 09/Jun/14 ] |
|
It's great to see a fix so quickly. Thanks for working on it! |
| Comment by Githook User [ 09/Jun/14 ] |
|
Author: {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}Message: |
| Comment by Asya Kamsky [ 31/May/14 ] |
|
Thank you for reporting this bug. EDIT Just noticed the reproducing script, so please disregard the previous request. I was able to reproduce the assert. Asya |