Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-32620

Allow moveChunk on unsharded collections



    • Task
    • Status: Backlog
    • Major - P3
    • Resolution: Unresolved
    • None
    • None
    • Sharding


      However, in 3.6, a chunk can only be moved if its estimated size is within the chunk size setting with a little bit of allowance, to prevent overflowing memory on the donor. Otherwise, the chunk will be marked as too big and the balancer will attempt to split it. If it fails to be split, then it is marked as jumbo. Since unsharded collections will have “allowSplit: false”, moveChunk will be changed to allow skipping the "chunk too big" check if allowSplit is false, as described below.

      The proposed changes in order to allow large chunks to be migrated without overflowing the memory on the donor shard constitute changes to the MigrationSourceManager to enter the critical section early, before the recipient starts the clone phase.
      One way to achieve this is by adding a parameter “ignoreJumbo” to the startClone method, which would cause it to not throw an error if the chunk is jumbo and perhaps to even skip the sorting phase.

      The MigrationSourceManager, will by default send startClone with ignoreJumbo set to false and if it receives a ChunkTooBig error, will check if this is the only chunk in the collection and allowSplit is false. If so, it will just enter the critical section immediately and call startClone again with ignoreJumbo set to true, but without registering the MigrationChunkClonerSource on the MigrationManager so that it won’t be tracking changes to the collection.


        Issue Links



              backlog-server-sharding Backlog - Sharding Team
              randolph@mongodb.com Randolph Tan
              1 Vote for this issue
              2 Start watching this issue