Here is a comment outlining the overall problem, plus the leading hypothesis as to WT's role in it. 

malikendsley/WT-16905 contains a fix that works (at least, based on the conclusion drawn in the comment) but I'm not sure if its idiomatic. I'm also not very familiar with WT's testing workflow. There's also a test on that branch in src/mongo/db/storage/wiredtiger/wiredtiger_c_api_test.cpp (SpillLikeModifyThenSearchHang) that repros the issue. Without the provisional fix on that branch, that unit test hangs.