Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-51248

Using _secondaryThrottle and _waitForDelete leads a high number of chunks

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Done
    • None
    • None
    • None
    • None
    • ALL

    Description

      Performing a synthetic workload I noticed a very different number of chunks and an error rate in the balancer when using the option:

      Below are the test results:

      # Default configuration
      mongos> sh.status(0 )mongos> sh.status(0 )
      --- Sharding Status ---
        sharding version:
      "_id" : 1"minCompatibleVersion" : 5"currentVersion" : 6"clusterId" : ObjectId("5f74ba2926ac7490e83e7c65")  }
        shards:
              {  "_id" : "shard01""host" : "shard01/localhost:37018,localhost:37019,localhost:37020""state" : 1 }
              {  "_id" : "shard02""host" : "shard02/localhost:37021,localhost:37022,localhost:37023""state" : 1 }
              {  "_id" : "shard03""host" : "shard03/localhost:37024,localhost:37025,localhost:37026""state" : 1 }
        active mongoses:        "4.0.20-13" : 1
        autosplit:
              Currently enabled: yes
        balancer:
              Currently enabled:  yes
              Currently running:  no 
             Failed balancer rounds in last 5 attempts:  0
              Migration Results for the last 24 hours:
                      16 : Success
                      129 : Failed with error 'aborted', from shard01 to shard03
        databases:
              {  "_id" : "config""primary" : "config""partitioned" : true }
                      config.system.sessions
                              shard key: { "_id" : 1 }
                              unique: false
                              balancing: true
                              chunks:
                                     shard01 1
                              { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard01 Timestamp(1, 0)
              {  "_id" : "vinnie""primary" : "shard01""partitioned" : true"version" : {  "uuid" : UUID("4bd88f33-481a-47d2-86c6-6a1fad5ed677"),  "lastMod" : 1 } }
                      vinnie.companies
                              shard key: { "_id" : 1 }
                              unique: false
                              balancing: true
                              chunks:
                                      shard01 10
                                      shard02 9
                                      shard03 9
                              too many chunks to print, use verbose if you want to force print
      mongos> db.companies.count()
      4813948
      

       And with the settings in place: 

      # Using the config below
      use config;
      db.settings.save( { _id:"chunksize", value: 64 });
      db.settings.save( { "_id" : "balancer", "_secondaryThrottle" : { "w" : "majority" }, "_waitForDelete" : true} )
      

      mongos> sh.status()
      --- Sharding Status ---
        sharding version: {   "_id" : 1"minCompatibleVersion" : 5"currentVersion" : 6"clusterId" : ObjectId("5f74ccd17a58008967dd6fbf")  }
        shards:
              {  "_id" : "shard01""host" : "shard01/localhost:37018,localhost:37019,localhost:37020""state" : 1 }
              {  "_id" : "shard02""host" : "shard02/localhost:37021,localhost:37022,localhost:37023""state" : 1 }
              {  "_id" : "shard03""host" : "shard03/localhost:37024,localhost:37025,localhost:37026""state" : 1 }
        active mongoses:        "4.0.20-13" : 1
        autosplit:
              Currently enabled: yes  balancer:
              Currently enabled:  yes
              Currently running:  no
              Failed balancer rounds in last 5 attempts:  0
              Migration Results for the last 24 hours:
                      64 : Success
        databases:
              {  "_id" : "config""primary" : "config""partitioned" : true }
                      config.system.sessions
                              shard key: { "_id" : 1 }
                              unique: false
                              balancing: true
                              chunks:
                                      shard01 1
                             { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard01 Timestamp(1, 0)
              {  "_id" : "vinnie""primary" : "shard01""partitioned" : true"version" : {  "uuid" : UUID("94232603-84da-43e0-bef7-783db0019696"),  "lastMod" : 1 } }
                      vinnie.companies
                              shard key: { "_id" : 1 }
                              unique: false
                              balancing: true
                              chunks:
                                      shard01 33
                                      shard02 34
                                      shard03 34
                              too many chunks to print, use verbose if you want to force print
      mongos> db.companies.count()
      4442950
      

      It seems like an issue since Mongo "gives up" about splitting the chunk and I'm afraid this can lead to jumbo chunks. On the other hand, with the settings enabled, I can observe an increased load on the shards (probably due to the write concern majority).

      Attachments

        1. template.json
          0.8 kB
          VINICIUS GRIPPA

        Activity

          People

            eric.sedor@mongodb.com Eric Sedor
            vgrippa@gmail.com VINICIUS GRIPPA
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: