[DOCS-9441] Docs for SERVER-26579: Sharding balancer moves chunks to shards whose utilization is above the cluster average Created: 02/Dec/16  Updated: 30/Oct/23  Resolved: 17/Mar/23

Status: Closed
Project: Documentation
Component/s: Server
Affects Version/s: None
Fix Version/s: Server_Docs_20231030

Type: Task Priority: Major - P3
Reporter: Emily Hall Assignee: Ravind Kumar (Inactive)
Resolution: Won't Do Votes: 0
Labels: DF
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
documents SERVER-26579 Sharding balancer moves chunks to sha... Closed
Participants:
Days since reply: 46 weeks, 6 days ago

 Description   

Engineering Ticket Description:

The current sharding balancer policy attempts to move chunks from shards which have more than the cluster-wide per-shard chunk count average to shards which have less than it. However there is a bug due to which chunks will move from highly loaded to less-loaded shard if the difference between the two is sufficiently large.

In particular consider the following case:

Shard1 - 1000
Shard2 - 900
Shard3 - 900
Shard4 - 800
Shard5 - 0
Shard6 - 0

Intuitively, this should produce only two migrations: Shard1 -> Shard5 and Shard2 -> Shard 6. However it will also cause a third migration from Shard3 -> Shard4, because 900 is above the ideal per-shard chunk number of 600.

Via kaloian.manassiev
> With this change, the balancer will now also attempt to move chunks out of shards whose max size setting is exceeded.



 Comments   
Comment by Sarah Olson [ 17/Mar/23 ]

Closing this out on the grounds that:

  • Details of this request are significantly outdated. 
  • Our documentation has evolved substantially since this request was made. 

Based on this, closing as WON'T DO. Please don't hesitate to give me a shout or to reopen if you disagree.

Comment by Kaloian Manassiev [ 31/Dec/16 ]

The balancer has no means to look into the "read/write utilization" - it only deals with number of chunks and estimated size of the chunk, using the formula that you pointed out. With SERVER-26579 the balancer will move chunks out of shards, whose number of chunks is higher than the average for the cluster / number of shards. Please have a look at the example in the ticket description.

The second bulled above is correct.

Comment by Ravind Kumar (Inactive) [ 02/Dec/16 ]

kaloian.manassiev I don't quite follow what has been resolved here.

From the summary and your comment, I think the change overall is that the balancer can move chunks off of a shard if either

  • The chunk has excessive / greater utilization (reads/writes?), moving it to a shard with less utilization overall
  • The number of chunks * chunksize > maxSize, so the balancer starts moving chunks until the shard size no longer validates maxSize.
Generated at Thu Feb 08 07:58:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.