Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-11328

Allow concurrent draining shards

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.2.1, 4.3.1
    • Component/s: Sharding
    • Labels:
    • Environment:
      All
    • Backwards Compatibility:
      Fully Compatible
    • Backport Requested:
      v4.2
    • Sprint:
      Sharding 2019-08-12, Sharding 2019-08-26
    • Case:

      Description

      Currently only one shard can be removed at a time. I trust that this was changed for a good reason but I feel a distinction should be made between actively draining a shard (which was problematic to do for more than one shard at the same time apparently) and tagging a shard as draining.

      The reason being is currently if we want to remove 5 shards for example and when the first of those 5 shards is draining it will potentially move its chunks to one of the other 4 shards that we want to remove anyway. This makes our cluster down scales unnecessarily slow since it's needlessly moving the same chunks around multiple times.

      The proposed solution is as follows :

      • Allow marking multiple shards as "draining"
      • Balancer picks one of these draining shards as the one that is going to be drained completely
      • Balancer avoids moving chunks to shards that are marked as "draining"

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              kaloian.manassiev Kaloian Manassiev
              Reporter:
              remonvv Remon van Vliet
              Participants:
              Votes:
              4 Vote for this issue
              Watchers:
              23 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: