Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-20392

Sharding an existing small collection results in large number of chunks

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Done
    • 3.0.6, 3.2.11, 3.4.0
    • 3.4.9, 3.5.12
    • Sharding
    • None
    • Minor Change
    • ALL
    • v3.4
    • Hide

      To reproduce:

      • set up a sharded cluster (1 single node shard is enough)
      • create a collection and fill it 20000 small documents containing only an int
      • shard the collection
      • insert a few more documents through the same mongos (the number will depend on the size of each document)
        A multi-split occurs, generating a high number of small chunks.

      Attached is a javascript file with these steps.

      Show
      To reproduce: set up a sharded cluster (1 single node shard is enough) create a collection and fill it 20000 small documents containing only an int shard the collection insert a few more documents through the same mongos (the number will depend on the size of each document) A multi-split occurs, generating a high number of small chunks. Attached is a javascript file with these steps.
    • Sharding 2017-01-02, Sharding 2017-07-31

    Description

      Creating a small collection (about < 4M) and then sharding the collection triggers a multi-split generating a large number of chunks (possibly thousands).

      Some examples:
      For 20000 initial documents containing only an ObjectId and an int, inserting 7 additional similar documents will trigger a split into 1819 chunks.
      For 90000 initial documents containing only an ObjectId and an int, inserting 7 additional similar documents will trigger a split into 8183 chunks.
      A hard limit is reached at 8192 and with 100000 + 7 documents, no split occurs.

      The behavior was observed with the following variations:

      • enabling/disabling the balancer
      • monotonically increasing shard key / hashed shard key
      • 1 or 2 shards

      The behavior did not occur on 2.6.11: only a few chunks are created (3 in the case of 20007 documents described before).

      Attachments

        Issue Links

          Activity

            People

              kevin.pulo@mongodb.com Kevin Pulo
              yves.duhem Yves Duhem
              Votes:
              5 Vote for this issue
              Watchers:
              28 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: