-
Type:
Bug
-
Resolution: Duplicate
-
Priority:
Major - P3
-
None
-
Affects Version/s: 3.6.2
-
Component/s: None
-
Labels:None
-
ALL
We have a three node MongoDB replica set deployed in our Prod environment. The primary mongod process gets hung up after running for 12 hours or so. We are able to see too many threads (around 15,000) stuck in the same stack,
#0 0x00007efd9c853c21 in do_futex_wait () from /lib64/libpthread.so.0 #1 0x00007efd9c853ce7 in __new_sem_wait_slow () from /lib64/libpthread.so.0 #2 0x00007efd9c853d85 in sem_timedwait () from /lib64/libpthread.so.0 #3 0x00005580f6d94c6c in mongo::TicketHolder::waitForTicketUntil(mongo::Date_t) () #4 0x00005580f681aedc in mongo::LockerImpl<false>::_lockGlobalBegin(mongo::LockMode, mongo::Duration<std::ratio<1l, 1000l> >) () #5 0x00005580f680a724 in mongo::Lock::GlobalLock::_enqueue(mongo::LockMode, unsigned int) () #6 0x00005580f680a79e in mongo::Lock::GlobalLock::GlobalLock(mongo::OperationContext*, mongo::LockMode, unsigned int, mongo::Lock::GlobalLock::EnqueueOnly) () #7 0x00005580f680a7e8 in mongo::Lock::GlobalLock::GlobalLock(mongo::OperationContext*, mongo::LockMode, unsigned int) ()
Attaching the pstack and diagnostic metrics.
Due to the sensitive nature of the db.logs they cannot be shared. The db logs had statements that showed around 14484 connections were open.
2020-05-22T13:38:49.633+0000 I NETWORK [listener] connection accepted from 172.28.96.186:43437 #18988 (14484 connections now open)
- related to
-
WT-3972 Allow more than 64K cursors to be open on a data source simultaneously
-
- Closed
-