[SERVER-59806] Optimized path of shard collection fails to succeed with a large number of chunks Created: 06/Sep/21  Updated: 29/Oct/23  Resolved: 10/Nov/21

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 5.2.0, 5.0.5, 5.1.1

Type: Bug Priority: Major - P3
Reporter: Marcos José Grillo Ramirez Assignee: Marcos José Grillo Ramirez
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Problem/Incident
causes SERVER-72301 Use ordered:false when inserting chun... Closed
is caused by SERVER-52779 Implement the new create/shard collec... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.1, v5.0
Steps To Reproduce:

1. Run a shard collection with a hashed shard key and 8000 chunks

Sprint: Sharding EMEA 2021-11-01, Sharding EMEA 2021-11-15
Participants:

 Description   

When sharding an empty collection, it is possible to pre-split the chunks across the cluster depending on the split policy initially associated with the collection. For example, if the shard key is hashed and an initial number of chunks is specified (using the numInitialChunks parameter) then the chunks will be equally distributed among the shards in the cluster automatically, given that the collection is empty.

When committing the chunks to the config server, an upsert is used, this works fine for a small number of chunks (for example a thousand) but, with a large enough number, it is making one node on the config server to crash. The upsert operation could be replaced by an insert, but then the limitation will be the size of the BSONObj on the command.



 Comments   
Comment by Githook User [ 15/Nov/21 ]

Author:

{'name': 'Marcos Jose Grillo Ramirez', 'email': 'marcos.grillo@mongodb.com', 'username': 'm4nti5'}

Message: SERVER-59806 Change upsert for insert when writing chunks in create sharded collection path

(cherry picked from commit bdc2e9b2ed299f4dfbf6183eed94707afbde8478)
Branch: v5.1
https://github.com/mongodb/mongo/commit/52e6fb7e647fbad710ec78bd641e0074a4b895a5

Comment by Githook User [ 15/Nov/21 ]

Author:

{'name': 'Marcos Jose Grillo Ramirez', 'email': 'marcos.grillo@mongodb.com', 'username': 'm4nti5'}

Message: SERVER-59806 Change upsert for insert when writing chunks in create sharded collection path

(cherry picked from commit bdc2e9b2ed299f4dfbf6183eed94707afbde8478)
Branch: v5.0
https://github.com/mongodb/mongo/commit/4da342b293978544e1f17da5dcccb47211c298bb

Comment by Githook User [ 10/Nov/21 ]

Author:

{'name': 'Marcos Jose Grillo Ramirez', 'email': 'marcos.grillo@mongodb.com', 'username': 'm4nti5'}

Message: SERVER-59806 Change upsert for insert when writing chunks in create sharded collection path
Branch: master
https://github.com/mongodb/mongo/commit/bdc2e9b2ed299f4dfbf6183eed94707afbde8478

Generated at Thu Feb 08 05:48:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.