[SERVER-59806] Optimized path of shard collection fails to succeed with a large number of chunks Created: 06/Sep/21 Updated: 29/Oct/23 Resolved: 10/Nov/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | None |
| Fix Version/s: | 5.2.0, 5.0.5, 5.1.1 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Marcos José Grillo Ramirez | Assignee: | Marcos José Grillo Ramirez |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Operating System: | ALL | ||||||||||||||||
| Backport Requested: |
v5.1, v5.0
|
||||||||||||||||
| Steps To Reproduce: | 1. Run a shard collection with a hashed shard key and 8000 chunks |
||||||||||||||||
| Sprint: | Sharding EMEA 2021-11-01, Sharding EMEA 2021-11-15 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
When sharding an empty collection, it is possible to pre-split the chunks across the cluster depending on the split policy initially associated with the collection. For example, if the shard key is hashed and an initial number of chunks is specified (using the numInitialChunks parameter) then the chunks will be equally distributed among the shards in the cluster automatically, given that the collection is empty. When committing the chunks to the config server, an upsert is used, this works fine for a small number of chunks (for example a thousand) but, with a large enough number, it is making one node on the config server to crash. The upsert operation could be replaced by an insert, but then the limitation will be the size of the BSONObj on the command. |
| Comments |
| Comment by Githook User [ 15/Nov/21 ] |
|
Author: {'name': 'Marcos Jose Grillo Ramirez', 'email': 'marcos.grillo@mongodb.com', 'username': 'm4nti5'}Message: (cherry picked from commit bdc2e9b2ed299f4dfbf6183eed94707afbde8478) |
| Comment by Githook User [ 15/Nov/21 ] |
|
Author: {'name': 'Marcos Jose Grillo Ramirez', 'email': 'marcos.grillo@mongodb.com', 'username': 'm4nti5'}Message: (cherry picked from commit bdc2e9b2ed299f4dfbf6183eed94707afbde8478) |
| Comment by Githook User [ 10/Nov/21 ] |
|
Author: {'name': 'Marcos Jose Grillo Ramirez', 'email': 'marcos.grillo@mongodb.com', 'username': 'm4nti5'}Message: |