[SERVER-16802] Order of balancer chunk moves depends on order of config.collections Created: 12/Jan/15 Updated: 30/May/18 Resolved: 16/Mar/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | 2.8.0-rc4 |
| Fix Version/s: | 3.4.15, 3.6.4, 3.7.4 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Kevin Pulo | Assignee: | Kevin Pulo |
| Resolution: | Done | Votes: | 1 |
| Labels: | balancer | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Backport Requested: |
v3.6, v3.4
|
||||||||
| Sprint: | Sharding 2018-02-26, Sharding 2018-03-12, Sharding 2018-03-26 | ||||||||
| Participants: | |||||||||
| Case: | (copied to CRM) | ||||||||
| Description |
BackgroundCurrently, Balancer::_doBalanceRound() and BalancerPolicy::balance() together find candidate chunks with the following pseudocode:
ProblemThis approach means that the balancer can attempt "regular" imbalance chunk moves before tag violation chunk moves, which can be before shard drain moves. This is counter-intuitive, because users expect:
ImpactThis is worsened by:
SuggestionPerhaps the code could be rearranged along the lines of:
Each collection would still have at most 1 chunk move per balance round, which should allow lower-priority moves to make progress (e.g. the balancer won't be "hogged" when draining a large shard, or when adding new tags). But it would also have the benefit of ensuring that — irrespective of the order of config.collections — draining moves are given priority over tag violation moves, which are given priority over imbalance moves. |
| Comments |
| Comment by Githook User [ 26/Mar/18 ] |
|
Author: {'email': 'kevin.pulo@mongodb.com', 'name': 'Kevin Pulo', 'username': 'devkev'}Message: (cherry picked from commit 651b3e017ce880d9ddbebb400af621c61d8c7389) |
| Comment by Githook User [ 19/Mar/18 ] |
|
Author: {'email': 'kevin.pulo@mongodb.com', 'name': 'Kevin Pulo', 'username': 'devkev'}Message: (cherry picked from commit 9850f1f190f13fb5bfd229e35d55d8fee3adc58f) |
| Comment by Githook User [ 16/Mar/18 ] |
|
Author: {'email': 'kevin.pulo@mongodb.com', 'name': 'Kevin Pulo', 'username': 'devkev'}Message: |
| Comment by Kaloian Manassiev [ 16/May/16 ] |
|
With the changes to move the balancer to the CSRS primary we split the balancer loop into a "policy" part, which returns list of all chunks, which need to be moved on that round. Currently this list is still sorted based on the collection namespace, but it would be fairly easy to make it sorted additionally first based on the draining status, then based on the tag violation status and finally on the collection name. |