Priority: Major - P3
Affects Version/s: 3.0.4
Fix Version/s: None
We have a sharding setup with tag-aware sharding and 22 shards. Each shard is a replica set consisting of 2 hosts and a backup replica (3 nodes in repset config)
We are facing issues where our mongod process is hitting 100% CPU usage on 1 thread and everything slows down.
We use tag-aware sharding to spread collections between shards.
Thus we have shards which only have data for specific database/collections.
We wonder, why we see querys (removes and updates) on shards which have no data for this database/collection. (All found in slow query log)
Due to this remove extra querys on 1 of our shards the lock "scheduler" went crazy and CPU usage where at 100% (1 thread) while IOPS and mongod operations got stalled.
1) Is it by design that remove commands are sent to all shards in a cluster even databse/collections doesn't exists on all shards?
2) Why do we see update querys on shards which don't have this database/collection?