Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-20392

Sharding an existing small collection results in large number of chunks

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 3.0.6, 3.2.11, 3.4.0
    • Fix Version/s: 3.4.9, 3.5.12
    • Component/s: Sharding
    • Labels:
      None
    • Backwards Compatibility:
      Minor Change
    • Operating System:
      ALL
    • Backport Requested:
      v3.4
    • Steps To Reproduce:
      Hide

      To reproduce:

      • set up a sharded cluster (1 single node shard is enough)
      • create a collection and fill it 20000 small documents containing only an int
      • shard the collection
      • insert a few more documents through the same mongos (the number will depend on the size of each document)
        A multi-split occurs, generating a high number of small chunks.

      Attached is a javascript file with these steps.

      Show
      To reproduce: set up a sharded cluster (1 single node shard is enough) create a collection and fill it 20000 small documents containing only an int shard the collection insert a few more documents through the same mongos (the number will depend on the size of each document) A multi-split occurs, generating a high number of small chunks. Attached is a javascript file with these steps.
    • Sprint:
      Sharding 2017-01-02, Sharding 2017-07-31
    • Case:

      CRM plugin field not viewable

      Description

      Creating a small collection (about < 4M) and then sharding the collection triggers a multi-split generating a large number of chunks (possibly thousands).

      Some examples:
      For 20000 initial documents containing only an ObjectId and an int, inserting 7 additional similar documents will trigger a split into 1819 chunks.
      For 90000 initial documents containing only an ObjectId and an int, inserting 7 additional similar documents will trigger a split into 8183 chunks.
      A hard limit is reached at 8192 and with 100000 + 7 documents, no split occurs.

      The behavior was observed with the following variations:

      • enabling/disabling the balancer
      • monotonically increasing shard key / hashed shard key
      • 1 or 2 shards

      The behavior did not occur on 2.6.11: only a few chunks are created (3 in the case of 20007 documents described before).

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                5 Vote for this issue
                Watchers:
                28 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: