Address sanitiser has detected a heap use after free:
(gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x00007f318dbd5921 in __GI_abort () at abort.c:79 #2 0x0000558d5865c397 in __sanitizer::Abort() () at /data/mci/2b29d50424b17e9bd64c23288b01e972/toolchain-builder/tmp/build-llvm.sh-h2X/llvm/projects/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cc:157 #3 0x0000558d5865ade1 in __sanitizer::Die() () at /data/mci/2b29d50424b17e9bd64c23288b01e972/toolchain-builder/tmp/build-llvm.sh-h2X/llvm/projects/compiler-rt/lib/sanitizer_common/sanitizer_termination.cc:59 #4 0x0000558d58642dc9 in ~ScopedInErrorReport () at /data/mci/2b29d50424b17e9bd64c23288b01e972/toolchain-builder/tmp/build-llvm.sh-h2X/llvm/projects/compiler-rt/lib/asan/asan_report.cc:187 #5 0x0000558d586445b3 in ReportGenericError () at /data/mci/2b29d50424b17e9bd64c23288b01e972/toolchain-builder/tmp/build-llvm.sh-h2X/llvm/projects/compiler-rt/lib/asan/asan_report.cc:464 #6 0x0000558d58644ebb in __asan_report_load8 () at /data/mci/2b29d50424b17e9bd64c23288b01e972/toolchain-builder/tmp/build-llvm.sh-h2X/llvm/projects/compiler-rt/lib/asan/asan_rtl.cc:121 #7 0x0000558d5b40d534 in __wt_btree_bytes_evictable (session=<optimized out>) at src/third_party/wiredtiger/src/include/btree_inline.h:142 #8 __session_dhandle_sweep (session=<optimized out>) at src/third_party/wiredtiger/src/session/session_dhandle.c:384 #9 __session_get_dhandle (session=<optimized out>, uri=<optimized out>, checkpoint=<optimized out>) at src/third_party/wiredtiger/src/session/session_dhandle.c:439 #10 __wt_session_get_dhandle (session=0x7f317ed31050, uri=0x607000575b60 "table:collection-1023--1887297523114510167", checkpoint=0x0, cfg=0x0, flags=0) at src/third_party/wiredtiger/src/session/session_dhandle.c:474 #11 0x0000558d5b396050 in __wt_schema_get_table_uri (session=0x7f317ed31050, uri=0x7f314d00f8f0 "\003B\200", ok_incomplete=false, flags=0, tablep=0x7f314d010c30) at src/third_party/wiredtiger/src/schema/schema_list.c:27 #12 0x0000558d5b1fbfcd in __wt_curtable_open (session=0x7f317ed31050, uri=0x607000575b60 "table:collection-1023--1887297523114510167", owner=0x0, cfg=0x7f314d010fa0, cursorp=0x7f314d010f80) at src/third_party/wiredtiger/src/cursor/cur_table.c:990 #13 0x0000558d5b3b6177 in __session_open_cursor_int (session=0x7f317ed31050, uri=0x607000575b60 "table:collection-1023--1887297523114510167", owner=0x0, other=0x0, cfg=0x7f314d010fa0, cursorp=0x7f314d010f80) at src/third_party/wiredtiger/src/session/session_api.c:453 #14 0x0000558d5b3b8783 in __session_open_cursor (wt_session=0x7f317ed31050, uri=0x607000575b60 "table:collection-1023--1887297523114510167", to_dup=0x0, config=0x7f3100000000 <error: Cannot access memory at address 0x7f3100000000>, cursorp=0x7f314d011320) at src/third_party/wiredtiger/src/session/session_api.c:603
Looking at the coredump at frame 8, the heap use after free shows that a dhandle is pointed at undefined memory:
(gdb) list 379 TAILQ_FOREACH_SAFE(dhandle_cache, &session->dhandles, q, dhandle_cache_tmp) 380 { 381 dhandle = dhandle_cache->dhandle; 382 empty_btree = false; 383 if (dhandle->type == WT_DHANDLE_TYPE_BTREE) 384 WT_WITH_DHANDLE( 385 session, dhandle, empty_btree = (__wt_btree_bytes_evictable(session) == 0)); 386 387 if (dhandle != session->dhandle && dhandle->session_inuse == 0 && 388 (WT_DHANDLE_INACTIVE(dhandle) || (gdb) print *dhandle Cannot access memory at address 0x8 (gdb) f 7 #7 0x0000558d5b40d534 in __wt_btree_bytes_evictable (session=<optimized out>) at src/third_party/wiredtiger/src/include/btree_inline.h:142 142 bytes_root = root_page == NULL ? 0 : root_page->memory_footprint; (gdb) print btree->dhandle->name $20 = 0x60400142d510 "file:index-1063--1887297523114510167.wt"
The address sanitiser has information about how the dhandle is created and freed.
Freeing stacktrace:
[j0:prim] 0x6070004c7a68 is located 40 bytes inside of 80-byte region [0x6070004c7a40,0x6070004c7a90) [j0:prim] freed by thread T13 here: [j0:prim] #0 0x558d5863e462 in free /data/mci/2b29d50424b17e9bd64c23288b01e972/toolchain-builder/tmp/build-llvm.sh-h2X/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:124:3 [j0:prim] #1 0x558d5afc8a32 in __wt_page_out /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/btree/bt_discard.c:133:5 [j0:prim] #2 0x558d5b2153eb in __wt_evict_file /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/evict/evict_file.c:106:13 [j0:prim] #3 0x558d5b0de604 in __wt_conn_dhandle_close /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/conn/conn_dhandle.c:401:9 [j0:prim] #4 0x558d5b0ff1ff in __sweep_discard_trees /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/conn/conn_sweep.c:161:9 [j0:prim] #5 0x558d5b0ff1ff in __sweep_server /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/conn/conn_sweep.c:323 [j0:prim] #6 0x7f318d7756da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
Allocating stacktrace:
[j0:prim] previously allocated by thread T1141 (conn1090) here: [j0:prim] #0 0x558d5863e9da in calloc /data/mci/2b29d50424b17e9bd64c23288b01e972/toolchain-builder/tmp/build-llvm.sh-h2X/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:155:3 [j0:prim] #1 0x558d5b2f5620 in __wt_calloc /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/os_common/os_alloc.c:50:14 [j0:prim] #2 0x558d5afe3451 in __wt_page_alloc /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/btree/bt_page.c:63:5 [j0:prim] #3 0x558d5afe49c0 in __wt_page_inmem /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/btree/bt_page.c:187:5 [j0:prim] #4 0x558d5afd64d9 in __wt_btree_tree_open /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/btree/bt_handle.c:697:5 [j0:prim] #5 0x558d5afcb5df in __wt_btree_open /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/btree/bt_handle.c:150:13 [j0:prim] #6 0x558d5b0df990 in __wt_conn_dhandle_open /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/conn/conn_dhandle.c:531:9 [j0:prim] #7 0x558d5b40c6d2 in __wt_session_get_dhandle /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/session/session_dhandle.c:510:20 [j0:prim] #8 0x558d5b40d24c in __wt_session_get_dhandle /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/session/session_dhandle.c:503:13 [j0:prim] #9 0x558d5b40b383 in __wt_session_get_btree_ckpt /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/session/session_dhandle.c:319:11 [j0:prim] #10 0x558d5b144a2b in __wt_curfile_open /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/include/time_inline.h [j0:prim] #11 0x558d5b3b5c39 in __session_open_cursor_int /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/session/session_api.c:487:13 [j0:prim] #12 0x558d5b1fc12f in __wt_curtable_open /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/cursor/cur_table.c:1000:15 [j0:prim] #13 0x558d5b3b6176 in __session_open_cursor_int /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/session/session_api.c:453:13 [j0:prim] #14 0x558d5b3b8782 in __session_open_cursor /data/mci/13e69e2ed46658a72d1a2c0c7c95251e/src/src/third_party/wiredtiger/src/session/session_api.c:603:5
The sweep server is looking for dead or excess dhandles to discard, and decides to free this particular dhandle. Furthermore, the stacktrace performs a cursor open on a dhandle that we have previously discarded, producing an invalid state, where wiredtiger is trying to open up a cursor on a dhandle that has been freed already.