AddressSanitizer: heap-use-after-free issue in JS setup/teardown

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Critical - P2
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Engines, Storage Engines - Foundations
    • SE Foundations - 2025-10-10
    • 5

      In SLS-2875, we are creating an AUBSAN build variant to run disagg suites with sanitizers. In testing the new variant, we seem to be running into a recurring issue:

      [j1:prim]     #0 0x55675583e1c2 in strlen /data/mci/6fc617b00a4218b9970b209d4517c50b/toolchain-builder/tmp/build-llvm-v5.sh-02L/llvm-project-llvmorg/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:391:5
[j1:prim]     #1 0x7fefda723b00 in __wti_cursor_set_valuev /proc/self/cwd/src/third_party/wiredtiger/src/cursor/cur_std.c:669:14
[j1:prim]     #2 0x7fefda72250f in __wt_cursor_set_value /proc/self/cwd/src/third_party/wiredtiger/src/cursor/cur_std.c:629:5
[j1:prim]     #3 0x7fefda5f5ab5 in __wt_disagg_update_shared_metadata /proc/self/cwd/src/third_party/wiredtiger/src/conn/conn_layered.c:2002:5
[j1:prim]     #4 0x7fefda3753f7 in __wti_block_disagg_checkpoint_resolve /proc/self/cwd/src/third_party/wiredtiger/src/block_disagg/block_disagg_ckpt.c:156:9
[j1:prim]     #5 0x7fefda56de37 in __checkpoint_tree /proc/self/cwd/src/third_party/wiredtiger/src/checkpoint/checkpoint_txn.c:2618:9
[j1:prim]     #6 0x7fefda579b9f in __checkpoint_tree_helper /proc/self/cwd/src/third_party/wiredtiger/src/checkpoint/checkpoint_txn.c:2698:11
[j1:prim]     #7 0x7fefda5798c4 in __checkpoint_apply_to_dhandles /proc/self/cwd/src/third_party/wiredtiger/src/checkpoint/checkpoint_txn.c:340:9
[j1:prim]     #8 0x7fefda572857 in __checkpoint_db_internal /proc/self/cwd/src/third_party/wiredtiger/src/checkpoint/checkpoint_txn.c:1343:5
[j1:prim]     #9 0x7fefda56af42 in __checkpoint_db_wrapper /proc/self/cwd/src/third_party/wiredtiger/src/checkpoint/checkpoint_txn.c:1734:11
[j1:prim]     #10 0x7fefda56a357 in __wt_checkpoint_db /proc/self/cwd/src/third_party/wiredtiger/src/checkpoint/checkpoint_txn.c:1813:9
[j1:prim]     #11 0x7fefda9995aa in __session_checkpoint /proc/self/cwd/src/third_party/wiredtiger/src/session/session_api.c:2363:11
[j1:prim]     #12 0x7fefdb121840 in decltype(auto) mongo::WiredTigerSession::checkpoint<char const (&) [19]>(char const (&) [19]) /proc/self/cwd/src/mongo/db/storage/wiredtiger/wiredtiger_session.h:134:5
[j1:prim]     #13 0x7fefdb1089d1 in mongo::WiredTigerKVEngine::_checkpoint(mongo::WiredTigerSession&, bool) /proc/self/cwd/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp:2118:29
[j1:prim]     #14 0x7fefdb0f3cab in mongo::WiredTigerKVEngine::_checkpoint(mongo::WiredTigerSession&) /proc/self/cwd/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp:2216:9
[j1:prim]     #15 0x7fefdb109b50 in mongo::WiredTigerKVEngine::checkpoint() /proc/self/cwd/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp:2231:12
[j1:prim]     #16 0x7fefdff11a0f in mongo::Checkpointer::run() /proc/self/cwd/src/mongo/db/storage/checkpointer.cpp:135:57
[j1:prim]     #17 0x7feff6b03304 in mongo::BackgroundJob::jobBody() /proc/self/cwd/src/mongo/util/background.cpp:156:5
[j1:prim]     #18 0x7feff6b070b1 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<mongo::stdx::thread::thread<mongo::BackgroundJob::go()::$_0, 0>(mongo::BackgroundJob::go()::$_0)::'lambda'()>>>::_M_run() /proc/self/cwd/src/mongo/util/background.cpp:183:13
[j1:prim]     #19 0x7feff41e2993 in execute_native_thread_routine unscaledcycleclock.cc
[j1:prim]     #20 0x5567558c32a6 in asan_thread_start(void*) /data/mci/6fc617b00a4218b9970b209d4517c50b/toolchain-builder/tmp/build-llvm-v5.sh-02L/llvm-project-llvmorg/compiler-rt/lib/asan/asan_interceptors.cpp:239:28
[j1:prim]     #21 0x7fefc1c8a4a9 in start_thread (/lib64/libc.so.6+0x8a4a9) (BuildId: 95cfbadda322f602c59deefb9f03d3c383d6a8ed)
[j1:prim]     #22 0x7fefc1d0f50f in __GI___clone3 (/lib64/libc.so.6+0x10f50f) (BuildId: 95cfbadda322f602c59deefb9f03d3c383d6a8ed)
[j1:prim]
[j1:prim] 0x51b000122f80 is located 0 bytes inside of 1442-byte region [0x51b000122f80,0x51b000123522)
[j1:prim] freed by thread T123 here:
[j1:prim]     #0 0x5567558c5656 in free /data/mci/6fc617b00a4218b9970b209d4517c50b/toolchain-builder/tmp/build-llvm-v5.sh-02L/llvm-project-llvmorg/compiler-rt/lib/asan/asan_malloc_linux.cpp:52:3
[j1:prim]     #1 0x7fefda7351b4 in __wt_cursor_copy_release_item /proc/self/cwd/src/third_party/wiredtiger/src/include/buf_inline.h:122:5
[j1:prim]     #2 0x7fefda65e931 in __cursor_copy_release /proc/self/cwd/src/third_party/wiredtiger/src/include/cursor_inline.h:74:13
[j1:prim]     #3 0x7fefda66f53f in __curfile_reset /proc/self/cwd/src/third_party/wiredtiger/src/cursor/cur_file.c:271:5
[j1:prim]     #4 0x7fefda83dff6 in __wt_metadata_cursor_release /proc/self/cwd/src/third_party/wiredtiger/src/meta/meta_table.c
[j1:prim]     #5 0x7fefda37527e in __wti_block_disagg_checkpoint_resolve /proc/self/cwd/src/third_party/wiredtiger/src/block_disagg/block_disagg_ckpt.c:143:5
[j1:prim]     #6 0x7fefda56de37 in __checkpoint_tree /proc/self/cwd/src/third_party/wiredtiger/src/checkpoint/checkpoint_txn.c:2618:9
[j1:prim]     #7 0x7fefda579b9f in __checkpoint_tree_helper /proc/self/cwd/src/third_party/wiredtiger/src/checkpoint/checkpoint_txn.c:2698:11
[j1:prim]     #8 0x7fefda5798c4 in __checkpoint_apply_to_dhandles /proc/self/cwd/src/third_party/wiredtiger/src/checkpoint/checkpoint_txn.c:340:9
[j1:prim]     #9 0x7fefda572857 in __checkpoint_db_internal /proc/self/cwd/src/third_party/wiredtiger/src/checkpoint/checkpoint_txn.c:1343:5
[j1:prim]     #10 0x7fefda56af42 in __checkpoint_db_wrapper /proc/self/cwd/src/third_party/wiredtiger/src/checkpoint/checkpoint_txn.c:1734:11
[j1:prim]     #11 0x7fefda56a357 in __wt_checkpoint_db /proc/self/cwd/src/third_party/wiredtiger/src/checkpoint/checkpoint_txn.c:1813:9
[j1:prim]     #12 0x7fefda9995aa in __session_checkpoint /proc/self/cwd/src/third_party/wiredtiger/src/session/session_api.c:2363:11
[j1:prim]     #13 0x7fefdb121840 in decltype(auto) mongo::WiredTigerSession::checkpoint<char const (&) [19]>(char const (&) [19]) /proc/self/cwd/src/mongo/db/storage/wiredtiger/wiredtiger_session.h:134:5
[j1:prim]     #14 0x7fefdb1089d1 in mongo::WiredTigerKVEngine::_checkpoint(mongo::WiredTigerSession&, bool) /proc/self/cwd/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp:2118:29
[j1:prim]     #15 0x7fefdb0f3cab in mongo::WiredTigerKVEngine::_checkpoint(mongo::WiredTigerSession&) /proc/self/cwd/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp:2216:9
[j1:prim]     #16 0x7fefdb109b50 in mongo::WiredTigerKVEngine::checkpoint() /proc/self/cwd/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp:2231:12
[j1:prim]     #17 0x7fefdff11a0f in mongo::Checkpointer::run() /proc/self/cwd/src/mongo/db/storage/checkpointer.cpp:135:57
[j1:prim]     #18 0x7feff6b03304 in mongo::BackgroundJob::jobBody() /proc/self/cwd/src/mongo/util/background.cpp:156:5
[j1:prim]     #19 0x7feff6b070b1 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<mongo::stdx::thread::thread<mongo::BackgroundJob::go()::$_0, 0>(mongo::BackgroundJob::go()::$_0)::'lambda'()>>>::_M_run() /proc/self/cwd/src/mongo/util/background.cpp:183:13
[j1:prim]     #20 0x7feff41e2993 in execute_native_thread_routine unscaledcycleclock.cc

      This issue is happening on all executions of disagg suites on the build variant.

      Link to logs
      Link to draft PR with the build variant

            Assignee:
            Luke Chen
            Reporter:
            Xuerui Fa
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated: