[SERVER-22616] Make balancer architecture more scalable by allowing parallel chunk migrations Created: 15/Feb/16  Updated: 15/Feb/16  Resolved: 15/Feb/16

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: Yoni Douek Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-4355 Support multiple parallel chunk migra... Closed
Related
related to SERVER-9072 Prepare machinery for parallel migrat... Closed
Participants:

 Description   

1. Moving one chunk at a time is not a feature of an allegedly "linear scalable" product.

2. Waiting for chunks to be deleted on a target shard, before moving chunks of a totally different collection, is not a scalable design.
Note: I'm not referring to the wait which is controllable via waitForDelete, I'm referring to not starting at all - i.e. "can't accept new chunks because there are still X deletes from previous migration"

3. If you detect this behavior, why not move to the next chunk in the meantime? other shards can accept chunks. The current bottleneck for our system at the moment is the range deleter on a single shard. Until it finishes deleting, no balancing occurs in the entire cluster. This is ridicoulus.



 Comments   
Comment by Yoni Douek [ 15/Feb/16 ]

OK, consider solving 2 and 3 in the meantime. They are unrelated.

Comment by Ramon Fernandez Marina [ 15/Feb/16 ]

Hi yonido, this feature was already requested in SERVER-4355, so I'm closing this ticket as a duplicate. Feel free to watch SERVER-4355 for updates and vote for it.

Regards,
Ramón.

Generated at Thu Feb 08 04:00:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.