Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-22616

Make balancer architecture more scalable by allowing parallel chunk migrations

    XMLWordPrintableJSON

Details

    • Icon: New Feature New Feature
    • Resolution: Duplicate
    • Icon: Major - P3 Major - P3
    • None
    • None
    • Sharding
    • None

    Description

      1. Moving one chunk at a time is not a feature of an allegedly "linear scalable" product.

      2. Waiting for chunks to be deleted on a target shard, before moving chunks of a totally different collection, is not a scalable design.
      Note: I'm not referring to the wait which is controllable via waitForDelete, I'm referring to not starting at all - i.e. "can't accept new chunks because there are still X deletes from previous migration"

      3. If you detect this behavior, why not move to the next chunk in the meantime? other shards can accept chunks. The current bottleneck for our system at the moment is the range deleter on a single shard. Until it finishes deleting, no balancing occurs in the entire cluster. This is ridicoulus.

      Attachments

        Activity

          People

            Unassigned Unassigned
            yonido Yoni Douek
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: