[SERVER-75481] Consider ideal data size when balancing Created: 30/Mar/23  Updated: 01/Dec/23  Resolved: 06/Apr/23

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 7.0.0-rc0, 6.0.6, 6.3.0-rc2
Fix Version/s: 7.0.0-rc0, 6.0.6, 6.3.2

Type: Bug Priority: Major - P3
Reporter: Pierlauro Sciarelli Assignee: Pierlauro Sciarelli
Resolution: Fixed Votes: 0
Labels: sharding-wfbf-day
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Problem/Incident
is caused by SERVER-65816 Change balancer policy to balance on ... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.3, v6.0
Sprint: Sharding EMEA 2023-04-17
Participants:

 Description   

The data size aware balancing introduced in v6.0.3+ versions can potentially schedule migrations from shards under the optimal data size threshold.

The chunk-based balancing was avoiding that by scheduling migrations only from shards with a sufficiently high number of chunks .

Example
There are 500GB to spread across 5 shards (100GB per shard) currently distributed in the following way:

  • Shard0: 200GB
  • Shard1: 200GB
  • Shard2: 50GB
  • Shard3: 40GB
  • Shard4: 10GB
  • Shard5: 0GB

Next balancing round (currently):

  • Shard0 donates to shard5
  • Shard1 donates to shard4
  • Shard2 donates to shard3 (this is the bug -> there is no real need to donate from shard2 to shard3 because they both need to reach 100GB)

Next balancing round (after this fix):

  • Shard0 donates to shard5
  • Shard1 donates to shard4


 Comments   
Comment by Githook User [ 20/Apr/23 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-75481 Consider ideal data size when balancing
Branch: v6.0
https://github.com/mongodb/mongo/commit/8a7a9f9fc2b058fc1b433f4c70843a5c9e985ff4

Comment by Githook User [ 20/Apr/23 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-75481 Consider ideal data size when balancing
Branch: v6.3
https://github.com/mongodb/mongo/commit/2d1c0a3a6385bb00020374d4a8dafd08ec56c64b

Comment by Githook User [ 06/Apr/23 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-75481 Consider ideal data size when balancing
Branch: master
https://github.com/mongodb/mongo/commit/a23da5fc5610cdd1c7189c9c767aeb62cbf6fa01

Generated at Thu Feb 08 06:30:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.