-
Type:
Bug
-
Resolution: Done
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Labels:None
-
ALL
Performing a synthetic workload I noticed a very different number of chunks and an error rate in the balancer when using the option:
Below are the test results:
# Default configuration
mongos> sh.status(0 )mongos> sh.status(0 )
--- Sharding Status ---
sharding version:
{ "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("5f74ba2926ac7490e83e7c65") }
shards:
{ "_id" : "shard01", "host" : "shard01/localhost:37018,localhost:37019,localhost:37020", "state" : 1 }
{ "_id" : "shard02", "host" : "shard02/localhost:37021,localhost:37022,localhost:37023", "state" : 1 }
{ "_id" : "shard03", "host" : "shard03/localhost:37024,localhost:37025,localhost:37026", "state" : 1 }
active mongoses: "4.0.20-13" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
16 : Success
129 : Failed with error 'aborted', from shard01 to shard03
databases:
{ "_id" : "config", "primary" : "config", "partitioned" : true }
config.system.sessions
shard key: { "_id" : 1 }
unique: false
balancing: true
chunks:
shard01 1
{ "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard01 Timestamp(1, 0)
{ "_id" : "vinnie", "primary" : "shard01", "partitioned" : true, "version" : { "uuid" : UUID("4bd88f33-481a-47d2-86c6-6a1fad5ed677"), "lastMod" : 1 } }
vinnie.companies
shard key: { "_id" : 1 }
unique: false
balancing: true
chunks:
shard01 10
shard02 9
shard03 9
too many chunks to print, use verbose if you want to force print
mongos> db.companies.count()
4813948
And with the settings in place:
# Using the config below
use config;
db.settings.save( { _id:"chunksize", value: 64 });
db.settings.save( { "_id" : "balancer", "_secondaryThrottle" : { "w" : "majority" }, "_waitForDelete" : true} )
mongos> sh.status()
--- Sharding Status ---
sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("5f74ccd17a58008967dd6fbf") }
shards:
{ "_id" : "shard01", "host" : "shard01/localhost:37018,localhost:37019,localhost:37020", "state" : 1 }
{ "_id" : "shard02", "host" : "shard02/localhost:37021,localhost:37022,localhost:37023", "state" : 1 }
{ "_id" : "shard03", "host" : "shard03/localhost:37024,localhost:37025,localhost:37026", "state" : 1 }
active mongoses: "4.0.20-13" : 1
autosplit:
Currently enabled: yes balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
64 : Success
databases:
{ "_id" : "config", "primary" : "config", "partitioned" : true }
config.system.sessions
shard key: { "_id" : 1 }
unique: false
balancing: true
chunks:
shard01 1
{ "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard01 Timestamp(1, 0)
{ "_id" : "vinnie", "primary" : "shard01", "partitioned" : true, "version" : { "uuid" : UUID("94232603-84da-43e0-bef7-783db0019696"), "lastMod" : 1 } }
vinnie.companies
shard key: { "_id" : 1 }
unique: false
balancing: true
chunks:
shard01 33
shard02 34
shard03 34
too many chunks to print, use verbose if you want to force print
mongos> db.companies.count()
4442950
It seems like an issue since Mongo "gives up" about splitting the chunk and I'm afraid this can lead to jumbo chunks. On the other hand, with the settings enabled, I can observe an increased load on the shards (probably due to the write concern majority).