-
Type:
Bug
-
Resolution: Works as Designed
-
Priority:
Major - P3
-
None
-
Affects Version/s: 3.2.11
-
Component/s: WiredTiger
-
Labels:None
-
ALL
We have small data-set mongodb. All data is processed in memory.
Total data size is only about 10GB. User data size is only 40MB and oplog.rs is 10GB.
And wiredtiger cache size is 20GB. And all user documents are changed once in 30 seconds.
There's 50 ~ 300 queries / second of geowithin query only on primary.
The weired thing is primary's checkpoint is placed too often.
"transaction checkpoint currently running" metric is always 1 of "db.serverStatus().wiredTiger.transaction. Attached graphs are "transaction checkpoint currently running" metric's.
PRIMARY> db.serverStatus().wiredTiger.transaction
{
"number of named snapshots created" : 0,
"number of named snapshots dropped" : 0,
"transaction begins" : 1466266063,
"transaction checkpoint currently running" : 1,
"transaction checkpoint generation" : 89630,
"transaction checkpoint max time (msecs)" : 6884,
"transaction checkpoint min time (msecs)" : 399,
"transaction checkpoint most recent time (msecs)" : 2220,
"transaction checkpoint scrub dirty target" : 0,
"transaction checkpoint scrub time (msecs)" : 0,
"transaction checkpoint total time (msecs)" : 163694262,
"transaction checkpoints" : 89630,
"transaction checkpoints skipped because database was clean" : 1,
"transaction failures due to cache overflow" : 0,
"transaction fsync calls for checkpoint after allocating the transaction ID" : 89629,
"transaction fsync duration for checkpoint after allocating the transaction ID (usecs)" : 7725,
"transaction range of IDs currently pinned" : 648,
"transaction range of IDs currently pinned by a checkpoint" : 2295,
"transaction range of IDs currently pinned by named snapshots" : 0,
"transaction sync calls" : 0,
"transactions committed" : 506988831,
"transactions rolled back" : 958130556
}
SECONDARY> db.serverStatus().wiredTiger.transaction
{
"number of named snapshots created" : 0,
"number of named snapshots dropped" : 0,
"transaction begins" : 786350822,
"transaction checkpoint currently running" : 0,
"transaction checkpoint generation" : 2574,
"transaction checkpoint max time (msecs)" : 4007,
"transaction checkpoint min time (msecs)" : 527,
"transaction checkpoint most recent time (msecs)" : 3104,
"transaction checkpoint scrub dirty target" : 0,
"transaction checkpoint scrub time (msecs)" : 0,
"transaction checkpoint total time (msecs)" : 8154109,
"transaction checkpoints" : 2574,
"transaction checkpoints skipped because database was clean" : 0,
"transaction failures due to cache overflow" : 0,
"transaction fsync calls for checkpoint after allocating the transaction ID" : 2574,
"transaction fsync duration for checkpoint after allocating the transaction ID (usecs)" : 32661,
"transaction range of IDs currently pinned" : 0,
"transaction range of IDs currently pinned by a checkpoint" : 0,
"transaction range of IDs currently pinned by named snapshots" : 0,
"transaction sync calls" : 0,
"transactions committed" : 809047207,
"transactions rolled back" : 454837
}
And disk writes are happend every 2 seconds on Primary but not secondary.
PRIMARY Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.00 24.00 0.00 417.00 0.00 18656.50 89.48 0.26 0.63 0.00 0.63 0.02 0.90 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 23.00 0.00 414.00 0.00 18783.00 90.74 0.36 0.87 0.00 0.87 0.02 0.90 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 23.00 0.00 408.00 0.00 18630.00 91.32 0.30 0.74 0.00 0.74 0.02 0.70 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 19.00 0.00 415.00 0.00 19192.50 92.49 0.31 0.75 0.00 0.75 0.02 0.80 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 28.00 0.00 399.00 0.00 18652.50 93.50 0.30 0.74 0.00 0.74 0.03 1.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 19.00 0.00 395.00 0.00 18685.00 94.61 0.30 0.75 0.00 0.75 0.02 0.70 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 27.00 0.00 399.00 0.00 18653.00 93.50 0.32 0.79 0.00 0.79 0.02 0.70 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 20.00 0.00 395.00 0.00 18951.00 95.95 0.31 0.78 0.00 0.78 0.02 0.80 sda 0.00 1.00 0.00 7.00 0.00 39.50 11.29 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 28.00 0.00 408.00 0.00 18655.50 91.45 0.32 0.79 0.00 0.79 0.02 0.80 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 19.00 0.00 414.00 0.00 18833.50 90.98 0.27 0.64 0.00 0.64 0.02 0.80 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 4.00 0.00 91.00 0.00 1468.00 32.26 0.03 0.29 0.00 0.29 0.01 0.10 sda 0.00 26.00 0.00 310.00 0.00 17557.00 113.27 0.25 0.81 0.00 0.81 0.02 0.70 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 18.00 0.00 400.00 0.00 18834.00 94.17 0.29 0.72 0.00 0.72 0.02 0.90 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 27.00 0.00 392.00 0.00 18665.00 95.23 0.32 0.82 0.00 0.82 0.02 0.80 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 18.00 0.00 397.00 0.00 18595.00 93.68 0.27 0.67 0.00 0.67 0.03 1.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 27.00 0.00 386.00 0.00 18501.00 95.86 0.30 0.77 0.00 0.77 0.02 0.80 sda 0.00 1.00 0.00 7.00 0.00 32.00 9.14 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 19.00 0.00 427.00 0.00 18889.50 88.48 0.30 0.70 0.00 0.70 0.02 0.90 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 26.00 0.00 413.00 0.00 18801.00 91.05 0.31 0.75 0.00 0.75 0.02 0.80 sda 0.00 0.00 0.00 2.00 0.00 16.00 16.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 24.00 0.00 450.00 0.00 19001.00 84.45 0.38 0.84 0.00 0.84 0.02 0.90 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 SECONDARY Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 4.00 0.00 28.00 14.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 1.00 0.00 3.00 0.00 10.00 6.67 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 1.00 0.00 6.50 13.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 2.00 0.00 8.00 8.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 23.00 0.00 2396.00 0.00 61823.00 51.61 8.10 3.38 0.00 3.38 0.01 3.20 sda 0.00 0.00 0.00 2.00 0.00 5.00 5.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 1.00 0.00 4.00 8.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
- is duplicated by
-
-
- Closed
-