[SERVER-49346] Re-enable use of libunwind with TSAN Created: 07/Jul/20  Updated: 14/Dec/22  Resolved: 14/Dec/22

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Ryan Egesdahl (Inactive) Assignee: Alex Neben
Resolution: Won't Fix Votes: 0
Labels: post-v4-toolchain
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-49078 Suppress ThreadSanitizer: signal-unsa... Closed
Assigned Teams:
Server Development Platform
Operating System: ALL
Steps To Reproduce:

Example backtrace:

>>> bt
#0  0x00007ffff4e7b727 in raise () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x00005555556462cc in __interceptor_raise () at /data/mci/76345995686809862f0b80d49f4a9e31/toolchain-builder/tmp/build-llvm.sh-IHB/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:1998
#2  0x00005555556bacb2 in mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}::operator()() const (this=0x7b0c00000c10) at src/mongo/util/stacktrace_test.cpp:435
#3  0x00005555556ba981 in std::__invoke_impl<void, mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}>(std::__invoke_other, mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}&&) (__f=...) at /opt/mongodbtoolchain/revisions/e5348beb43e147b74a40f4ca5fb05a330ea646cf/stow/gcc-v3.U0D/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/invoke.h:60
#4  0x00005555556ba921 in std::__invoke<mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}>(std::__invoke_result&&, (mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}&&)...) (__fn=...) at /opt/mongodbtoolchain/revisions/e5348beb43e147b74a40f4ca5fb05a330ea646cf/stow/gcc-v3.U0D/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/invoke.h:95
#5  0x00005555556ba87d in std::__apply_impl<mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}, std::tuple<>>(mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}&&, std::tuple<>&&, std::integer_sequence<unsigned long>) (__f=..., __t=...) at /opt/mongodbtoolchain/revisions/e5348beb43e147b74a40f4ca5fb05a330ea646cf/stow/gcc-v3.U0D/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/tuple:1678
#6  0x00005555556ba80e in std::apply<mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}, std::tuple<> >(mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}&&, std::tuple<>&&) (__f=..., __t=...) at /opt/mongodbtoolchain/revisions/e5348beb43e147b74a40f4ca5fb05a330ea646cf/stow/gcc-v3.U0D/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/tuple:1687
#7  0x00005555556ba751 in mongo::stdx::thread::thread<mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}, , 0>(mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2})::{lambda()#1}::operator()() (this=0x7b0c00000c08) at src/mongo/stdx/thread.h:186
#8  0x00005555556ba6c1 in std::__invoke_impl<void, mongo::stdx::thread::thread<mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}, , 0>(mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2})::{lambda()#1}>(std::__invoke_other, mongo::stdx::thread::thread<mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}, , 0>(mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2})::{lambda()#1}&&) (__f=...) at /opt/mongodbtoolchain/revisions/e5348beb43e147b74a40f4ca5fb05a330ea646cf/stow/gcc-v3.U0D/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/invoke.h:60
#9  0x00005555556ba5e1 in std::__invoke<mongo::stdx::thread::thread<mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}, , 0>(mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2})::{lambda()#1}> (__fn=...) at /opt/mongodbtoolchain/revisions/e5348beb43e147b74a40f4ca5fb05a330ea646cf/stow/gcc-v3.U0D/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/bits/invoke.h:95
#10 0x00005555556ba599 in std::thread::_Invoker<std::tuple<mongo::stdx::thread::thread<mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}, , 0>(mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2})::{lambda()#1}> >::_M_invoke<0ul> (this=0x7b0c00000c08) at /opt/mongodbtoolchain/revisions/e5348beb43e147b74a40f4ca5fb05a330ea646cf/stow/gcc-v3.U0D/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/thread:234
#11 0x00005555556ba549 in std::thread::_Invoker<std::tuple<mongo::stdx::thread::thread<mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}, , 0>(mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2})::{lambda()#1}> >::operator()() (this=0x7b0c00000c08) at /opt/mongodbtoolchain/revisions/e5348beb43e147b74a40f4ca5fb05a330ea646cf/stow/gcc-v3.U0D/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/thread:243
#12 0x00005555556ba2cd in std::thread::_State_impl<std::thread::_Invoker<std::tuple<mongo::stdx::thread::thread<mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2}, , 0>(mongo::(anonymous namespace)::StackTraceSigAltStackTest::tryHandler(void (*)(int, siginfo_t*, void*))::{lambda()#2})::{lambda()#1}> > >::_M_run() (this=0x7b0c00000c00) at /opt/mongodbtoolchain/revisions/e5348beb43e147b74a40f4ca5fb05a330ea646cf/stow/gcc-v3.U0D/lib/gcc/x86_64-mongodb-linux/8.2.0/../../../../include/c++/8.2.0/thread:186
#13 0x00005555557247df in std::execute_native_thread_routine (__p=0x7b0c00000c00) at ../../../../../src/combined/libstdc++-v3/src/c++11/thread.cc:80
#14 0x000055555563fa03 in __tsan_thread_start_func () at /data/mci/76345995686809862f0b80d49f4a9e31/toolchain-builder/tmp/build-llvm.sh-IHB/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:955
#15 0x00007ffff4e706db in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#16 0x00007ffff498188f in clone () from /lib/x86_64-linux-gnu/libc.so.6

Participants:

 Description   

We recently discovered in SERVER-49078 that attempting to trigger a signal handler during the stacktrace unit tests. There isn't an obvious reason in the code why this should be happening, but it prevents the stacktrace tests from passing when run with TSAN right now.



 Comments   
Comment by Alex Neben [ 14/Dec/22 ]

{"t":{"$date":"2022-12-14T19:27:34.924Z"},"s":"I",  "c":"NETWORK",  "id":4915701, "ctx":"main","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":20},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":20},"outgoing":{"minWireVersion":6,"maxWireVersion":20},"isInternalClient":false}}}
{"t":{"$date":"2022-12-14T19:27:34.926Z"},"s":"I",  "c":"TEST",     "id":23063,   "ctx":"main","msg":"Running","attr":{"suite":"JsonTest"}}
{"t":{"$date":"2022-12-14T19:27:34.927Z"},"s":"I",  "c":"TEST",     "id":23059,   "ctx":"main","msg":"Running","attr":{"test":"Hex","rep":1,"reps":1}}
{"t":{"$date":"2022-12-14T19:27:34.927Z"},"s":"I",  "c":"TEST",     "id":23060,   "ctx":"main","msg":"Done running tests"}
{"t":{"$date":"2022-12-14T19:27:34.927Z"},"s":"I",  "c":"TEST",     "id":23063,   "ctx":"main","msg":"Running","attr":{"suite":"PrintAllThreadStacksTest"}}
{"t":{"$date":"2022-12-14T19:27:34.927Z"},"s":"I",  "c":"TEST",     "id":23059,   "ctx":"main","msg":"Running","attr":{"test":"WithDeadThreads","rep":1,"reps":1}}
{"t":{"$date":"2022-12-14T19:27:34.943Z"},"s":"I",  "c":"COMMAND",  "id":23394,   "ctx":"thread1","msg":"Preparing to dump thread stacks","attr":{"numThreads":6}}
{"t":{"$date":"2022-12-14T19:27:34.943Z"},"s":"I",  "c":"COMMAND",  "id":23396,   "ctx":"thread1","msg":"Signalled threads","attr":{"numThreads":6}}
[1]    20346 segmentation fault  ./build/install/bin/stacktrace_test 

Thread 1 "stacktrace_test" received signal SIGUSR2, User defined signal 2. 0x00007ffff578d680 in nanosleep () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt #0  0x00007ffff578d680 in nanosleep () from /lib/x86_64-linux-gnu/libc.so.6 #1  0x00007ffff578d55a in sleep () from /lib/x86_64-linux-gnu/libc.so.6 #2  0x00005555555a880f in __interceptor_sleep () at /data/mci/a6d133bde62bf8bdef4fc3e1d412ae09/toolchain-builder/tmp/build-llvm-v4.sh-bpA/llvm-project-llvmorg/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:349 #3  0x00005555556264f4 in mongo::(anonymous namespace)::UnitTest_SuiteNamePrintAllThreadStacksTestTestNameWithDeadThreads::_doTest (this=0x7fffffffd240) at src/mongo/util/stacktrace_test.cpp:623 #4  0x00007ffff7f7f355 in mongo::unittest::Test::run (this=0x7fffffffd240) at src/mongo/unittest/unittest.cpp:244 #5  0x0000555555625b38 in mongo::unittest::Test::RegistrationAgent<mongo::(anonymous namespace)::UnitTest_SuiteNamePrintAllThreadStacksTestTestNameWithDeadThreads>::RegistrationAgent(mongo::unittest::TestInfo const*)::{lambda()#1}::operator()() const (this=<optimized out>) at src/mongo/unittest/framework.h:344 #6  std::__invoke_impl<void, mongo::unittest::Test::RegistrationAgent<mongo::(anonymous namespace)::UnitTest_SuiteNamePrintAllThreadStacksTestTestNameWithDeadThreads>::RegistrationAgent(mongo::unittest::TestInfo const*)::{lambda()#1}&>(std::__invoke_other, mongo::unittest::Test::RegistrationAgent<mongo::(anonymous namespace)::UnitTest_SuiteNamePrintAllThreadStacksTestTestNameWithDeadThreads>::RegistrationAgent(mongo::unittest::TestInfo const*)::{lambda()#1}&) (__f=...) at /opt/mongodbtoolchain/revisions/ecdb6f23a2cc2e822b8dc2eeb3f04c31f25c66ce/stow/gcc-v4.8nU/lib/gcc/x86_64-mongodb-linux/11.3.0/../../../../include/c++/11.3.0/bits/invoke.h:61 #7  std::__invoke_r<void, mongo::unittest::Test::RegistrationAgent<mongo::(anonymous namespace)::UnitTest_SuiteNamePrintAllThreadStacksTestTestNameWithDeadThreads>::RegistrationAgent(mongo::unittest::TestInfo const*)::{lambda()#1}&>(void&&, (mongo::unittest::Test::RegistrationAgent<mongo::(anonymous namespace)::UnitTest_SuiteNamePrintAllThreadStacksTestTestNameWithDeadThreads>::RegistrationAgent(mongo::unittest::TestInfo const*)::{lambda()#1}&)...) (__fn=...) at /opt/mongodbtoolchain/revisions/ecdb6f23a2cc2e822b8dc2eeb3f04c31f25c66ce/stow/gcc-v4.8nU/lib/gcc/x86_64-mongodb-linux/11.3.0/../../../../include/c++/11.3.0/bits/invoke.h:111 #8  std::_Function_handler<void (), mongo::unittest::Test::RegistrationAgent<mongo::(anonymous namespace)::UnitTest_SuiteNamePrintAllThreadStacksTestTestNameWithDeadThreads>::RegistrationAgent(mongo::unittest::TestInfo const*)::{lambda()#1}>::_M_invoke(std::_Any_data const&) (__functor=...) at /opt/mongodbtoolchain/revisions/ecdb6f23a2cc2e822b8dc2eeb3f04c31f25c66ce/stow/gcc-v4.8nU/lib/gcc/x86_64-mongodb-linux/11.3.0/../../../../include/c++/11.3.0/bits/std_function.h:290 #9  0x00007ffff7f81d89 in std::function<void ()>::operator()() const (this=0x7b48000001c0) at /opt/mongodbtoolchain/revisions/ecdb6f23a2cc2e822b8dc2eeb3f04c31f25c66ce/stow/gcc-v4.8nU/lib/gcc/x86_64-mongodb-linux/11.3.0/../../../../include/c++/11.3.0/bits/std_function.h:590 #10 mongo::unittest::Suite::run (this=<optimized out>, filter="", fileNameFilter=..., runsPerTest=<optimized out>, runsPerTest@entry=1) at src/mongo/unittest/unittest.cpp:439 #11 0x00007ffff7f840a6 in mongo::unittest::Suite::run (suites=std::vector of length 0, capacity 0, filter="", fileNameFilter="", runsPerTest=1) at src/mongo/unittest/unittest.cpp:508 #12 0x00007ffff7fb987b in main (argc=<optimized out>, argv=<optimized out>) at src/mongo/unittest/unittest_main.cpp:137 #13 0x00007ffff56cac87 in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6 #14 0x000055555558274a in _start () at /opt/mongodbtoolchain/revisions/ecdb6f23a2cc2e822b8dc2eeb3f04c31f25c66ce/stow/gcc-v4.8nU/lib/gcc/x86_64-mongodb-linux/11.3.0/../../../../include/c++/11.3.0/bits/std_function.h:244

python3 ./buildscripts/scons.py --variables-files= --variables-files=etc/scons/mongodbtoolchain_v4_clang.vars --dbg=on --opt=on --allocator=system --sanitize=thread --install-action=hardlink --implicit-cache --build-fast-and-loose=on --link-model=dynamic ICECC= CCACHE= build/install/bin/stacktrace_test 

Looks as though v4 toolchain did not fix this problem. Closing this ticket as running without libunwind for tsan is not a big deal.

Comment by Billy Donahue [ 16/Jul/20 ]

Ok sure.

Comment by Andrew Morrow (Inactive) [ 16/Jul/20 ]

billy.donahue - Should we leave this as the tracking ticket for "re-enable libunwind for TSAN"? We could backlog it into the epic for the v4 toolchain.

Comment by Billy Donahue [ 15/Jul/20 ]

SERVER-49078 was resolved by disabling libunwind when TSAN is active.

That happened after this ticket was forked from it.

This ticket might need clarification or closure now.

Generated at Thu Feb 08 05:19:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.