[SERVER-14306] mongos can cause the in-memory sort limit to be hit on shards by requesting more results than needed Created: 19/Jun/14 Updated: 19/Jun/15 Resolved: 18/Dec/14 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | 2.6.0, 2.8.0-rc2 |
| Fix Version/s: | 2.6.7, 2.8.0-rc4 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Daniel Pasette (Inactive) | Assignee: | Spencer Brody (Inactive) |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||
| Operating System: | ALL | ||||||||||||
| Backport Completed: | |||||||||||||
| Participants: | |||||||||||||
| Description |
| Comments |
| Comment by Githook User [ 23/Dec/14 ] | |||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}Message: | |||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 18/Dec/14 ] | |||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}Message: | |||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 18/Dec/14 ] | |||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}Message: | |||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by David Storch [ 11/Dec/14 ] | |||||||||||||||||||||||||||||||||||||||||||||||||
|
This issue still exists as of 2.8.0-rc2. Updated repro steps using mtools:
| |||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by David Storch [ 01/Jul/14 ] | |||||||||||||||||||||||||||||||||||||||||||||||||
|
After some further investigation, it looks like this not a duplicate of This issue arises when you have large documents, and thus cannot fit ntoreturn documents into a single batch. Say you have a sharded cluster with one shard. The shard has 40 documents, each about 1 MB in size. Then you deliver a query with .limit(10). The shard should be able to answer this query without hitting the 32 MB memory limit by doing a topK. Here is what ends up happening:
One way to fix this would be to have mongos update ntoreturn for each getmore. The original query would have ntoreturn==10. After getting back 4 results, the first getmore would have ntoreturn==6. Similarly, the second getmore would have ntoreturn==2. | |||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by David Storch [ 30/Jun/14 ] | |||||||||||||||||||||||||||||||||||||||||||||||||
|
Confirmed as a duplicate of |