Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-81100

Investigate Slow Splitting When Using sh.ShardCollection()

    XMLWordPrintableJSON

Details

    • Icon: Task Task
    • Resolution: Unresolved
    • Icon: Major - P3 Major - P3
    • None
    • 5.0.21
    • None
    • None
    • Sharding EMEA
    • Sharding EMEA 2023-10-16

    Description

      During recent testing balancer enhancements I found that initially sharding a collection with a small chunk size will significantly increase the duration to complete sharding collection relative to sharding an empty collection and loading the data with the same chunk size.

      Test Scenario - MongoDB 5.0
      1:

      • 4TB Collection Size
      • 32KB Doc Size
      • 1MB Chunk Size
      • Shard Existing Unsharded Collection with sh.shardCollection

      2:

      • Set 1MB Chunk Size
      • Shard Empty Collection with sh.shardCollection
      • Insert Documents
      • 32KB Doc Size
      • 1MB Chunk Size

      Scenario 1 will take much longer than Scenario 2

      Attachments

        Activity

          People

            matt.panton@mongodb.com Matt Panton
            matt.panton@mongodb.com Matt Panton
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: