[SERVER-83775] Do not balance data between shards owning more than the ideal data size Created: 01/Dec/23  Updated: 02/Jan/24  Resolved: 20/Dec/23

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 6.0.3, 7.0.0, 7.2.0
Fix Version/s: 7.2.1, 7.3.0-rc0, 6.0.13, 7.0.6

Type: Bug Priority: Major - P3
Reporter: Pierlauro Sciarelli Assignee: Pierlauro Sciarelli
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Problem/Incident
is caused by SERVER-65816 Change balancer policy to balance on ... Closed
Assigned Teams:
Catalog and Routing
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v7.2, v7.0, v6.0
Sprint: CAR Team 2023-12-25
Participants:
Story Points: 2

 Description   

The data size aware balancing introduced in v6.0.3+ versions can potentially schedule migrations between shards that are above the optimal data size threshold.

The chunk-based balancing was avoiding that by only scheduling migrations to shards with a sufficiently low number of chunks.

Example

There are 400GB to spread across 4 shards (ideally 100GB per shard) currently distributed in the following way:

  • Shard0: 130GB
  • Shard1: 120GB
  • Shard2: 110GB
  • Shard3: 40GB
Next balancing round (currently):
  • Shard0 donates to shard3
  • Shard1 donates to shard2 (this is the bug -> there is no real need to donate from shard1 to shard2 because they both need to reach 100GB)
Expected next balancing round after fixing this:
  • Shard0 donates to shard3

(Problem/solution very similar to SERVER-75481)



 Comments   
Comment by Githook User [ 29/Dec/23 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-83775 Do not balance data between shards owning more than the ideal data size
Branch: v7.2
https://github.com/mongodb/mongo/commit/3da705246cb770ed2864a135dd9868cf449342f4

Comment by Githook User [ 21/Dec/23 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-83775 Do not balance data between shards owning more than the ideal data size

GitOrigin-RevId: a30880ebcb903444463300e967809f257df13d10
Branch: v6.0
https://github.com/mongodb/mongo/commit/8962bb0b8bd131c3691f6e4e281972c6194af0e8

Comment by Githook User [ 21/Dec/23 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-83775 Do not balance data between shards owning more than the ideal data size

GitOrigin-RevId: 02b922dfc9a930e0e8f432f10d8f4d50c6a34500
Branch: v7.0
https://github.com/mongodb/mongo/commit/a7963df9648eb27ab96ad5255157f5b371bdecc9

Comment by Githook User [ 20/Dec/23 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-83775 Do not balance data between shards owning more than the ideal data size

GitOrigin-RevId: d68e62491c9eab5300b498a304f6601fd4dca79f
Branch: master
https://github.com/mongodb/mongo/commit/fdeeea2044cffa538252e98ef808a89951b9ad2e

Generated at Thu Feb 08 06:53:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.