-
Type:
Bug
-
Resolution: Done
-
Priority:
Major - P3
-
None
-
Affects Version/s: 3.0.7
-
Component/s: WiredTiger
-
Labels:None
-
Fully Compatible
-
ALL
Our mongo installation is experiencing weird issue: after starting mongod, CPU usage on one core starts growing, until it's hitting 100% (on just one core, with other CPU cores mostly idle).
We're observing this on secondaries and on the primary.
Restarting mongod on the affected server "fixes" the issue - for a few hours hours, CPU usage is very small on all cores, then, it gradually grows to 100% on one core. After a few days mongod is barely responsive, even simple operations like "show dbs" are taking 10 seconds or more.
The issue is not related to excessive disk IO or network traffic (which both remains very small).
When we run "perf top -g", it mostly shows the following:
Samples: 1M of event 'cpu-clock', Event count (approx.): 46746422538 65.90% [kernel] [k] xen_hypercall_sched_op 30.43% mongod [.] 0x00000000008e4ea8 0.73% [kernel] [k] xen_hypercall_xen_version 0.27% mongod [.] __wt_tree_walk 0.26% libstdc++.so.6.0.19 [.] std::_Rb_tree_increment(std::_Rb_tree_node_base*) 0.13% mongod [.] __wt_evict 0.11% libc-2.19.so [.] 0x000000000007fc89 0.11% perf [.] 0x0000000000056596 0.09% mongod [.] __wt_row_search 0.09% mongod [.] __wt_page_in_func 0.07% libstdc++.so.6.0.19 [.] std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::string const&) 0.06% mongod [.] snappy::RawUncompress(snappy::Source*, char*) (...)
Related mongodb-user discussion:
https://groups.google.com/forum/#!topic/mongodb-user/K2WkN2dOKMA
Let me know what other information you need.
