-
Type:
Improvement
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
-
Fully Compatible
-
Execution Team 2020-05-04
Most system threads use an IdleThreadBlock to hint to debuggers, including the hang analyzer, that it should not print its stacks because the thread is idle.
This would be a helpful improvement for developers looking at the hang analyzer output. Even though the WaitForMajorityService is often idle here, the stack trace looks suspicious.
As an example, this is what an idle FTDC thread looks like in the hang_analyzer output:
Thread 20: "ftdc" (Thread 0x7f3ab6d9b700 (LWP 4200)) Thread is idle at src/mongo/db/ftdc/controller.cpp:214
and this is what an idle WaitForMajorityService thread looks like:
Thread 19: "waitForMajority" (Thread 0x7f3ab819d700 (LWP 4198))
#0 0x00007f3ac8f12403 in poll () from /lib64/libc.so.6
#1 0x00007f3ace846da7 in mongo::transport::TransportLayerASIO::BatonASIO::run (this=0x7f3ad76c3610, clkSource=0x7f3ad23d49c0) at /opt/mongodbtoolchain/revisions/toolchain_builder_rhel62_patch_57df61bfe08e9c2805a38d201c5cd6c0ea414778_5daee4e830661503f51984bf_19_10_22_11_16_02/stow/gcc-v3.cZb/include/c++/8.2.0/bits/stl_vector.h:805
#2 0x00007f3aced423ae in mongo::Waitable::wait<mongo::BasicLockableAdapter>(mongo::Waitable*, mongo::ClockSource*, mongo::stdx::condition_variable&, mongo::BasicLockableAdapter&)::{lambda()#1}::operator()() const (this=<optimized out>) at src/mongo/util/lockable_adapter.h:57
#3 mongo::stdx::condition_variable::_runWithNotifyable<mongo::Waitable::wait<mongo::BasicLockableAdapter>(mongo::Waitable*, mongo::ClockSource*, mongo::stdx::condition_variable&, mongo::BasicLockableAdapter&)::{lambda()#1}>(mongo::Notifyable&, mongo::BasicLockableAdapter&&) (cb=..., notifyable=..., this=0x7f3ad29e2670) at src/mongo/stdx/condition_variable.h:162
#4 mongo::Waitable::wait<mongo::BasicLockableAdapter> (lk=..., cv=..., clkSource=<optimized out>, waitable=0x7f3ad76c3610) at src/mongo/util/waitable.h:61
#5 mongo::OperationContext::<lambda()>::operator() (__closure=<optimized out>) at src/mongo/db/operation_context.cpp:304
#6 mongo::OperationContext::waitForConditionOrInterruptNoAssertUntil (this=0x7f3ad76c3480, cv=..., m=..., deadline=...) at src/mongo/db/operation_context.cpp:309
#7 0x00007f3acd732367 in mongo::Interruptible::<lambda(mongo::Date_t, mongo::Interruptible::WakeSpeed)>::operator() (speed=mongo::Interruptible::WakeSpeed::kSlow, deadline=..., this=<optimized out>) at src/mongo/util/lockable_adapter.h:50
#8 mongo::Interruptible::<lambda(mongo::Date_t, mongo::Interruptible::WakeSpeed)>::operator() (speed=mongo::Interruptible::WakeSpeed::kSlow, deadline=..., this=<optimized out>) at src/mongo/util/interruptible.h:423
#9 mongo::Interruptible::waitForConditionOrInterruptUntil<std::unique_lock<mongo::latch_detail::Latch>, mongo::WaitForMajorityService::_periodicallyWaitForMajority(mongo::ServiceContext*)::<lambda()> > (waitTimer=0x0, pred=..., finalDeadline=..., m=..., cv=..., this=<optimized out>) at src/mongo/util/interruptible.h:448
#10 mongo::Interruptible::waitForConditionOrInterrupt<std::unique_lock<mongo::latch_detail::Latch>, mongo::WaitForMajorityService::_periodicallyWaitForMajority(mongo::ServiceContext*)::<lambda()> > (waitTimer=0x0, pred=..., m=..., cv=..., this=<optimized out>) at src/mongo/util/interruptible.h:466
#11 mongo::WaitForMajorityService::_periodicallyWaitForMajority (this=<optimized out>, service=<optimized out>) at src/mongo/db/s/wait_for_majority_service.cpp:184
#12 0x00007f3acd733bd0 in mongo::WaitForMajorityService::<lambda()>::operator() (__closure=0x7f3ad74769a0, __closure=0x7f3ad74769a0) at src/mongo/db/s/wait_for_majority_service.cpp:68