[SERVER-49503] No thread names printed in GDB from core dump on ARM spawn host Created: 14/Jul/20  Updated: 27/Oct/23  Resolved: 10/Nov/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Spencer Brody (Inactive) Assignee: Andrew Morrow (Inactive)
Resolution: Gone away Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Sprint: Dev Platform 2021-11-15
Participants:

 Description   

I made a spawn host with the data from this build failure: https://evergreen.mongodb.com/task/mongodb_mongo_v4.4_enterprise_ubuntu1804_arm64_noPassthrough_1_enterprise_ubuntu1804_arm64_946e1ca19f65925bfeb1551dad00b2d36246fcfe_20_07_10_02_35_55/0#/log/E (BF-18199)

 

When I attached GDB to the core dump, however, "info threads" does not print thread names.  Not sure if this is because this was an ARM build or something else, but that seems the most likely culprit.



 Comments   
Comment by Andrew Morrow (Inactive) [ 10/Nov/21 ]

Since this issue can now be worked around by using the v4 toolchain GDB, I'm closing this as gone away.

Comment by Andrew Morrow (Inactive) [ 10/Nov/21 ]

Now that we have the v4 GDB (GDB 11), I tried this again. I used 4.4.10-49-g00828e0 because as of SERVER-52821 the threadName variable no longer exists, so v4.4 is the last branch that has it.

Using the v4 GDB (GDB 8):

(gdb) p mongo::for_debuggers::threadName
No symbol "threadName" in namespace "mongo::for_debuggers".
 
(gdb) p ::mongo::for_debuggers::threadName
No symbol "threadName" in namespace "mongo::for_debuggers".
 
(gdb) p (mongo::StringData)_ZN5mongo13for_debuggers10threadNameE
$1 = {_data = 0xffffa18ce948 "initandlisten", _size = 13}

Which is consistent with the above. Using the v4 toolchain GDB (from toolchain revision 8391ec859b03d3bebb666821a88e616ae61a567a)we get:

(gdb) p mongo::for_debuggers::threadName
$1 = {_data = 0xffffa18ce948 "initandlisten", _size = 13}
 
(gdb) p ::mongo::for_debuggers::threadName
$2 = {_data = 0xffffa18ce948 "initandlisten", _size = 13}
 
(gdb) p (mongo::StringData)_ZN5mongo13for_debuggers10threadNameE
$3 = {_data = 0xffffa18ce948 "initandlisten", _size = 13}
 
(gdb) p ::mongo::for_debuggers::idleThreadLocation
$4 = 0xaaaaed5c5cb8 "src/mongo/db/db.cpp:794"

Comment by Andrew Morrow (Inactive) [ 11/Aug/20 ]

I'm bucketing this into the v4 toolchain. I can definitely reproduce it, but I can't see any way to fix it. I think it is probably a platform specific GDB (or glibc?) bug on ARM.

(gdb) p mongo::for_debuggers::threadName
No symbol "threadName" in namespace "mongo::for_debuggers".
 
(gdb) p ::mongo::for_debuggers::threadName
No symbol "threadName" in namespace "mongo::for_debuggers".
 
(gdb) p ::mongo::for_debuggers
Attempt to use a type name as an expression
 
(gdb) p (mongo::StringData)_ZN5mongo13for_debuggers10threadNameE
$1 = {_data = 0xffff750bf7f0 "ReplNetwork", _size = 11}
 
(gdb) p ::mongo::for_debuggers::idleThreadLocation
$2 = 0x0

So it for some reason can see idleThreadLocation but not threadName, in the same namespace, but it can see it via the mangled name. We should retest with a newer GDB when we upgrade the toolchain.

Comment by Spencer Brody (Inactive) [ 14/Jul/20 ]

Also note that running "p mongo::for_debuggers::threadName" within GDB didn't work, but running "p (mongo::StringData)_ZN5mongo13for_debuggers10threadNameE" did work, for getting the thread name for a specific thread at least. I got the mangled name from running "nm dist-test/bin/mongod| grep threadName|grep for_debuggers"

Generated at Thu Feb 08 05:20:02 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.