Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17393

Balancer lock not released

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: 3.0.0-rc10
    • Component/s: Sharding
    • Fully Compatible
    • ALL
    • Hide

      2 shard node replica set, all running wiredTiger
      3 config servers
      1 mongos
      Heavy insert workload, 4 clients

      Show
      2 shard node replica set, all running wiredTiger 3 config servers 1 mongos Heavy insert workload, 4 clients

      The balancer is shown as running with an active lock in over 20 hours.

      mongos> sh.status()
      --- Sharding Status ---
        sharding version: {
      	"_id" : 1,
      	"minCompatibleVersion" : 5,
      	"currentVersion" : 6,
      	"clusterId" : ObjectId("54edd63deae8d68b1c02b2bf")
      }
        shards:
      	{  "_id" : "shard0",  "host" : "shard0/ip-10-102-151-133:30000,ip-10-203-175-233:30000,ip-10-63-43-228:30000" }
      	{  "_id" : "shard1",  "host" : "shard1/ip-10-180-37-181:30000,ip-10-187-63-140:30000,ip-10-203-173-62:30000" }
        balancer:
      	Currently enabled:  yes
      	Currently running:  yes
      		Balancer lock taken at Wed Feb 25 2015 19:30:06 GMT+0000 (UTC) by ip-10-123-129-75:37017:1424873021:1804289383:Balancer:1681692777
      	Collections with active migrations:
      		test.cap1833hashed started at Wed Feb 25 2015 19:30:07 GMT+0000 (UTC)
      	Failed balancer rounds in last 5 attempts:  0
      	Migration Results for the last 24 hours:
      		32 : Success
      		1 : Failed with error '_recvChunkCommit failed!', from shard1 to shard0
      		3 : Failed with error 'chunk too big to move', from shard1 to shard0
      		247 : Failed with error 'could not acquire collection lock for test.cap1833hashed to migrate chunk [{ : MinKey },{ : MaxKey }) :: caused by :: Lock for migrating chunk [{ : MinKey }, { : MaxKey }) in test.cap1833hashed is taken.', from shard1 to shard0
        databases:
      	{  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }
      	{  "_id" : "test",  "partitioned" : true,  "primary" : "shard0" }
      		test.cap1833hashed
      			shard key: { "idx" : "hashed" }
      			chunks:
      				shard0	57324
      				shard1	64971
      			too many chunks to print, use verbose if you want to force print
      

        1. mongos-balancer-lock.log.gz
          8.50 MB
        2. configdb.tar.gz
          16.19 MB

            Assignee:
            randolph@mongodb.com Randolph Tan
            Reporter:
            jonathan.abrahams Jonathan Abrahams
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: