Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-44088

Autosplitter seems to ignore some fat chunks

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Incomplete
    • 4.0.10
    • None
    • None
    • None
    • ALL

    Description

      We have a somewhat large and hot MongoDB cluster. Size of each shard is ~400GiB. Autosplitter is configured to split chunks when they exceed 64 MB (default value). However, I see a lot of much larger chunks. Some chunks are as large as 400 MB. The chunks are in a hot collection that is constantly being inserted and updated. Distribution of hot and cold documents across chunks is even (we use hashed sharding on _id with auto generated ObjectId).

      As far as I can see, autosplitter is throttled by getting a token from the token pool with 5 tokens in it. I suspect that in a large shard with unevenly hot collections autosplitter activity is unfairly distributed in favor of hotter collections with a possibility of never getting autosplitter attention for colder collections.

      We've hit hard into this issue when added new shards to our cluster and balancer started moving chunks. When balancer spots chunk that is too big, it splits the chunk and balances smaller chunk to new shards. As a result, with evenly distributed chunks across the cluster we have 3x documents in older shards and 1x documents in newer shards.

      Attachments

        Issue Links

          Activity

            People

              eric.sedor@mongodb.com Eric Sedor
              sz Sergey Zagursky
              Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: