[SERVER-3569] Find operation does not return all documents that are previously inserted into a sharded environment while chunks are moved. Created: 10/Aug/11 Updated: 10/Dec/14 Resolved: 28/May/13 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | 1.8.2, 1.9.1, 2.0.0 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Michael Wagner | Assignee: | Randolph Tan |
| Resolution: | Duplicate | Votes: | 5 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Ubunutu 10.10 x86_64 |
||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Operating System: | ALL | ||||||||
| Participants: | |||||||||
| Description |
|
To test the mongoDb sharding capabilities we have continuously inserted data into a two shards environment. Each shard consists of These gaps are independent of the used write concern. To reproduce the problem a java maven project is attached to this issue:
Details:
|
| Comments |
| Comment by Scott Hernandez (Inactive) [ 22/Sep/11 ] |
|
The issue is the chunk distribution (based on the shard key) and not the number of shards. If there are few chunks, like the initial one when you shard the collection before there is much data, it will have to do lots of migration until the distribution of chunks is even among all shards. Even then if the shard key isn't well distributed there can be host spots/chunk will require more splits, and will cause the balancer to move them around, which will cause more of these issues. With a load and well distributed system this will be much, much less likely over time. You could manage this a bit by only doing balancing in maint. windows which will make this behavior more predictable. |
| Comment by Michael Wagner [ 22/Sep/11 ] |
|
We could reproduce this issue with mongoDb version 2.0.0. To create the test environment we have stopped all mongo processes, replaced the old binaries, performend a clean up upon all data and finally started all mongo processes again. |
| Comment by Michael Wagner [ 22/Sep/11 ] |
|
The issue also appears if both shards are already activated at the beginning of our test. |
| Comment by Scott Hernandez (Inactive) [ 22/Sep/11 ] |
|
That pretty much makes the issue one of chunk migration with the queued write-backs happening at the end of the migration. With a better balanced system, or more stable distribution of chunk, this should be less of an issue as compared to a simple test starting from an empty data-set. |
| Comment by Michael Wagner [ 22/Sep/11 ] |
|
The test environment is equal to the environment used to reproduce |
| Comment by Michael Wagner [ 22/Sep/11 ] |
|
In case the balancing is turned off the described issue does not appear. |
| Comment by Scott Hernandez (Inactive) [ 22/Sep/11 ] |
|
If balancing is turned off before running the tests, does the issue still appear? |