Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-42273

Introduce a "force" option to `moveChunk` to allow migrating jumbo chunks

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.3.1
    • Component/s: None
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Sprint:
      Sharding 2019-09-23, Sharding 2019-10-07, Sharding 2019-10-21, Sharding 2019-11-04, Sharding 2019-11-18

      Description

      Currently, if a chunk is larger than 64MB by default or 1GB max, the balancer will mark it as jumbo and will refuse to move it.

      It is possible to manually issue a moveChunk command and pass the unsupported and undocumented maxChunkSizeBytes parameter, which will override the check for max chunk size, but even with this, given sufficient write load to the chunk being migrated, the memory usage on the donor shard could exceed 500MB in which case migration will still fail.

      This ticket proposes adding a new forceJumbo option to the moveChunk command in order to allow large chunks to be migrated at the possible expense of blocking writes to the owning collection on the shard in question. The option will have the following deviation from the way it currently operates:

      1. It will skip the step, which sorts the cloned chunk's document ids and will instead give out the chunks in the order of the shard key (this means it will never return a 'jumbo chunk' error)
      2. Instead of failing the migration, if the memory usage exceeds 500MB, it will instead enter the critical section (this means that writes to the collection being migrated will possibly block for longer period of time)

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                1 Vote for this issue
                Watchers:
                17 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: