Details
-
Bug
-
Status: Closed
-
Major - P3
-
Resolution: Duplicate
-
2.2.1
-
None
-
None
-
None
-
Linux
Description
We have a sharded cluster of 5 mongod's. They are started like this:
dbpath = /mnt/mongo/rs0
|
shardsvr = true
|
maxConns = 20000
|
keyFile = keyfile
|
|
|
# Logging
|
logpath = logs/mongod-cn60.log
|
|
|
# Replica set
|
replSet = rs0
|
oplogSize = 1024
|
These are connected to by 96 nodes running mongos.
There are three config servers are run like:
dbpath = /mnt/mongo/cs0
|
configsvr = true
|
keyFile = keyfile
|
|
|
# Logging
|
logpath = configserver-cn60.log
|
We see the config server log file flooded with:
Wed Dec 5 18:51:44 [conn583] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn584] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn813] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn756] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn289] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn828] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn561] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn244] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn823] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn168] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn651] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn1286] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn630] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn773] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn230] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn583] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn756] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn813] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn584] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn1320] CMD fsync: sync:1 lock:0
|
Wed Dec 5 18:51:44 [conn126] CMD fsync: sync:1 lock:0
|
The config server has continuous 30% usage with no load on the cluster.
It could be that each mongos is trying to balance, and this causes the config server to be permanently loaded. Disabling balancing reduces the load considerably to 5-10% CPU (cluster still not loaded). However we still see this fsync command every 30s or so from every connection.
Is this expected? Can we enable balancing on only one mongos?
Attachments
Issue Links
- is duplicated by
-
SERVER-5442 use journal commit instead of fsync command for config server writes
-
- Closed
-