Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-20392

Sharding an existing small collection results in large number of chunks

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 3.4.9, 3.5.12
    • Affects Version/s: 3.0.6, 3.2.11, 3.4.0
    • Component/s: Sharding
    • Labels:
      None
    • Minor Change
    • ALL
    • v3.4
    • Hide

      To reproduce:

      • set up a sharded cluster (1 single node shard is enough)
      • create a collection and fill it 20000 small documents containing only an int
      • shard the collection
      • insert a few more documents through the same mongos (the number will depend on the size of each document)
        A multi-split occurs, generating a high number of small chunks.

      Attached is a javascript file with these steps.

      Show
      To reproduce: set up a sharded cluster (1 single node shard is enough) create a collection and fill it 20000 small documents containing only an int shard the collection insert a few more documents through the same mongos (the number will depend on the size of each document) A multi-split occurs, generating a high number of small chunks. Attached is a javascript file with these steps.
    • Sharding 2017-01-02, Sharding 2017-07-31

      Creating a small collection (about < 4M) and then sharding the collection triggers a multi-split generating a large number of chunks (possibly thousands).

      Some examples:
      For 20000 initial documents containing only an ObjectId and an int, inserting 7 additional similar documents will trigger a split into 1819 chunks.
      For 90000 initial documents containing only an ObjectId and an int, inserting 7 additional similar documents will trigger a split into 8183 chunks.
      A hard limit is reached at 8192 and with 100000 + 7 documents, no split occurs.

      The behavior was observed with the following variations:

      • enabling/disabling the balancer
      • monotonically increasing shard key / hashed shard key
      • 1 or 2 shards

      The behavior did not occur on 2.6.11: only a few chunks are created (3 in the case of 20007 documents described before).

        1. reproduce.js
          1 kB
          Yves Duhem

            Assignee:
            kevin.pulo@mongodb.com Kevin Pulo
            Reporter:
            yves.duhem Yves Duhem
            Votes:
            5 Vote for this issue
            Watchers:
            28 Start watching this issue

              Created:
              Updated:
              Resolved: