Consider ideal data size when balancing

XMLWordPrintableJSON

    • Fully Compatible
    • ALL
    • v6.3, v6.0
    • Sharding EMEA 2023-04-17
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The data size aware balancing introduced in v6.0.3+ versions can potentially schedule migrations from shards under the optimal data size threshold.

      The chunk-based balancing was avoiding that by scheduling migrations only from shards with a sufficiently high number of chunks .

      Example
      There are 500GB to spread across 5 shards (100GB per shard) currently distributed in the following way:

      • Shard0: 200GB
      • Shard1: 200GB
      • Shard2: 50GB
      • Shard3: 40GB
      • Shard4: 10GB
      • Shard5: 0GB

      Next balancing round (currently):

      • Shard0 donates to shard5
      • Shard1 donates to shard4
      • Shard2 donates to shard3 (this is the bug -> there is no real need to donate from shard2 to shard3 because they both need to reach 100GB)

      Next balancing round (after this fix):

      • Shard0 donates to shard5
      • Shard1 donates to shard4

              Assignee:
              Pierlauro Sciarelli
              Reporter:
              Pierlauro Sciarelli
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: