[SERVER-58281] hang analyzer unable to load BSON library Created: 05/Jul/21  Updated: 29/Oct/23  Resolved: 04/Jan/22

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 5.3.0

Type: Bug Priority: Major - P3
Reporter: Benety Goh Assignee: Tausif Rahman (Inactive)
Resolution: Fixed Votes: 0
Labels: neweng, tig-hanganalyzer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-56019 timeseries_index.js hangs in burn_in_... Closed
is related to SERVER-26634 Integrate GDB pretty printers for ser... Closed
is related to SERVER-58277 add minimal gdb pretty printer for op... Closed
Backwards Compatibility: Fully Compatible
Sprint: STM 2022-01-10
Participants:
Story Points: 2

 Description   

I first noticed this in a patch build debugging a timed out BF-related issue.

It seems that the hang analyzer script is unable to locate a usable Python BSON library when it installs the GDB pretty printers. Using a recent non-patch related task as an example, this prevents the hang analyzer from decoding BSONObj function arguments in the stack traces:

x debugger_mongod.logWarning: Could not load bson library for Python '3.9.2 (default, May 21 2021, 20:36:03)
Warning: Could not load bson library for Python '3.9.2 (default, May 21 2021, 20:36:03)
#22 0x00007f53c067f7e6 in mongo::repl::(anonymous namespace)::CmdHello::runWithReplyBuilder (this=0x7f53c068f2a0 <mongo::repl::(anonymous namespace)::cmdIsMaster>, opCtx=0x55e0f5da3540, cmdObj=owned BSONObj 120 bytes @ 0x55e0f656691d - unprintable or invalid, replyBuilder=0x55e0f6170960) at src/third_party/boost/boost/optional/optional.hpp:396
#22 0x00007fcb91e867e6 in mongo::repl::(anonymous namespace)::CmdHello::runWithReplyBuilder (this=0x7fcb91e962a0 <mongo::repl::(anonymous namespace)::cmdIsMaster>, opCtx=0x56003dcefc00, cmdObj=owned BSONObj 120 bytes @ 0x56003cf98d5d - unprintable or invalid, replyBuilder=0x56003df217c0) at src/third_party/boost/boost/optional/optional.hpp:396
 
x debugger_mongod_2241.log#22 0x00007fcb91e867e6 in mongo::repl::(anonymous namespace)::CmdHello::runWithReplyBuilder (this=0x7fcb91e962a0 <mongo::repl::(anonymous namespace)::cmdIsMaster>, opCtx=0x56003dcefc00, cmdObj=owned BSONObj 120 bytes @ 0x56003cf98d5d - unprintable or invalid, replyBuilder=0x56003df217c0) at src/third_party/boost/boost/optional/optional.hpp:396
 
x debugger_mongod_2241_raw_stacks.log#22 0x00007fcb91e867e6 in mongo::repl::(anonymous namespace)::CmdHello::runWithReplyBuilder (this=0x7fcb91e962a0 <mongo::repl::(anonymous namespace)::cmdIsMaster>, opCtx=0x56003dcefc00, cmdObj=owned BSONObj 120 bytes @ 0x56003cf98d5d - unprintable or invalid, replyBuilder=0x56003df217c0) at src/third_party/boost/boost/optional/optional.hpp:396
 
x debugger_mongod_471.log
x debugger_mongod_471_raw_stacks.log#22 0x00007f7f3a0f67e6 in mongo::repl::(anonymous namespace)::CmdHello::runWithReplyBuilder (this=0x7f7f3a1062a0 <mongo::repl::(anonymous namespace)::cmdIsMaster>, opCtx=0x559456e76c00, cmdObj=owned BSONObj 120 bytes @ 0x55945bf5321d - unprintable or invalid, replyBuilder=0x559456bd87c0) at src/third_party/boost/boost/optional/optional.hpp:396
 
x debugger_mongod_477.log#22 0x00007f53c067f7e6 in mongo::repl::(anonymous namespace)::CmdHello::runWithReplyBuilder (this=0x7f53c068f2a0 <mongo::repl::(anonymous namespace)::cmdIsMaster>, opCtx=0x55e0f5da3540, cmdObj=owned BSONObj 120 bytes @ 0x55e0f656691d - unprintable or invalid, replyBuilder=0x55e0f6170960) at src/third_party/boost/boost/optional/optional.hpp:396
 
x debugger_mongod_477_raw_stacks.log#22 0x00007f53c067f7e6 in mongo::repl::(anonymous namespace)::CmdHello::runWithReplyBuilder (this=0x7f53c068f2a0 <mongo::repl::(anonymous namespace)::cmdIsMaster>, opCtx=0x55e0f5da3540, cmdObj=owned BSONObj 120 bytes @ 0x55e0f656691d - unprintable or invalid, replyBuilder=0x55e0f6170960) at src/third_party/boost/boost/optional/optional.hpp:396



 Comments   
Comment by Githook User [ 04/Jan/22 ]

Author:

{'name': 'trahman1318', 'email': 'tausif.rahman@mongodb.com', 'username': 'trahman1318'}

Message: SERVER-58281 hang analyzer unable to load BSON library
Branch: master
https://github.com/mongodb/mongo/commit/2295d0afb02a5954e9ea359962e9537c89265c74

Comment by Robert Guo (Inactive) [ 06/Jul/21 ]

As Richard shrewdly pointed out, we don't seem to be using venv based on the output of sys.path in Benety's above comment.

Comment by Robert Guo (Inactive) [ 06/Jul/21 ]

Thanks for the detailed info Benety. We can take a first look.

For STM:
since this works on a spawn host, we should look into if there're anything weird about the physical hosts or environment that failed to load the BSON library, either because a previous task clobbered the package that GDB uses somehow or the incorrect Python or GDB is used.

Generated at Thu Feb 08 05:44:05 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.