-
Type:
Build Failure
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Storage Engines, Storage Engines - Persistence
-
SE Persistence backlog
-
None
-
2
format-stress-test-tsan on amazon2023-arm64-tsan
Host: i-0c226467f90d624f4
Project: wiredtiger
Commit: a57a4415
Please refer to BF(G) Playbook for instructions on handling BF and BFG tickets as well as Auto-Resolution Rules
Task Logs:
format-stress-test-tsan task_log
Logs:
WARNING: ThreadSanitizer: data race (pid=227272)
Read of size 4 at 0xffff94204054 by thread T184 (mutexes: write M0, write M1, write M2):
#0 __wt_log_get_backup_files /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/log/log.c:499:33 (libwiredtiger.so.12.0.0+0x4d0dec)
#1 __backup_log_append /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/cursor/cur_backup.c:529:9 (libwiredtiger.so.12.0.0+0x3625dc)
#2 __backup_start /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/cursor/cur_backup.c:872:9 (libwiredtiger.so.12.0.0+0x35e494)
#3 __wt_curbackup_open /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/cursor/cur_backup.c:394:9 (libwiredtiger.so.12.0.0+0x359b6c)
#4 __session_open_cursor_int /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/session/session_api.c:713:13 (libwiredtiger.so.12.0.0+0x656b08)
#5 __session_open_cursor /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/session/session_api.c:904:5 (libwiredtiger.so.12.0.0+0x65a21c)
#6 backup /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/test/format/backup.c:575:18 (t+0xfd7d8)
Previous atomic write of size 8 at 0xffff94204050 by thread T47 (mutexes: write M3):
#0 __wt_atomic_store_uint64_relaxed /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/include/gcc.h:398:1 (libwiredtiger.so.12.0.0+0x506b5c)
#1 __log_slot_close /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/log/log_slot.c:187:5 (libwiredtiger.so.12.0.0+0x50ad50)
#2 __log_slot_switch_internal /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/log/log_slot.c:352:15 (libwiredtiger.so.12.0.0+0x50781c)
#3 __wti_log_slot_switch /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/log/log_slot.c:413:9 (libwiredtiger.so.12.0.0+0x506ea0)
#4 __wti_log_force_write /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/log/log.c:2496:13 (libwiredtiger.so.12.0.0+0x4cf638)
#5 __log_server /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/log/log_mgr.c:972:13 (libwiredtiger.so.12.0.0+0x502ca4)
Location is heap block of size 15976 at 0xffff94204000 allocated by main thread:
#0 calloc /data/mci/f49c90f0be13703fc74cd438c017c2d0/toolchain-builder/tmp/build-llvm-v5.sh-WmZ/llvm-project-llvmorg/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:686:5 (t+0x6c2a4)
#1 __wt_calloc /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/os_common/os_alloc.c:68:14 (libwiredtiger.so.12.0.0+0x533ee0)
#2 __wt_logmgr_create /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/log/log_mgr.c:1056:5 (libwiredtiger.so.12.0.0+0x50112c)
#3 __wti_connection_workers /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/conn/conn_open.c:258:5 (libwiredtiger.so.12.0.0+0x33cc30)
#4 wiredtiger_open /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/conn/conn_api.c:3531:5 (libwiredtiger.so.12.0.0+0x2dff00)
#5 wts_open /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/test/format/wts.c:841:9 (t+0x143818)
#6 main /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/test/format/t.c:358:9 (t+0x134714)
Mutex M0 (0xffff8f754b00) created at:
#0 pthread_mutex_init /data/mci/f49c90f0be13703fc74cd438c017c2d0/toolchain-builder/tmp/build-llvm-v5.sh-WmZ/llvm-project-llvmorg/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1327:3 (t+0x6feb4)
#1 __wt_spin_init /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/include/mutex_inline.h:142:5 (libwiredtiger.so.12.0.0+0x689d88)
#2 __open_session /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/session/session_api.c:2602:5 (libwiredtiger.so.12.0.0+0x65bbc4)
#3 __wt_open_session /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/session/session_api.c:2727:5 (libwiredtiger.so.12.0.0+0x65b454)
#4 __conn_open_session /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/conn/conn_api.c:1399:5 (libwiredtiger.so.12.0.0+0x2e79ec)
#5 wt_wrap_open_session /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/test/format/format_util.c:380:5 (t+0x1166c4)
#6 backup /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/test/format/backup.c:472:5 (t+0xfd190)
Mutex M1 (0xffff9900a8f0) created at:
#0 pthread_mutex_init /data/mci/f49c90f0be13703fc74cd438c017c2d0/toolchain-builder/tmp/build-llvm-v5.sh-WmZ/llvm-project-llvmorg/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1327:3 (t+0x6feb4)
#1 __wt_spin_init /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/include/mutex_inline.h:142:5 (libwiredtiger.so.12.0.0+0x31a0d4)
#2 __wti_connection_init /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/conn/conn_handle.c:50:5 (libwiredtiger.so.12.0.0+0x318e74)
#3 wiredtiger_open /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/conn/conn_api.c:3120:5 (libwiredtiger.so.12.0.0+0x2dc118)
#4 wts_open /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/test/format/wts.c:841:9 (t+0x143818)
#5 main /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/test/format/t.c:358:9 (t+0x134714)
Mutex M2 (0xffff9900aaa0) created at:
#0 pthread_mutex_init /data/mci/f49c90f0be13703fc74cd438c017c2d0/toolchain-builder/tmp/build-llvm-v5.sh-WmZ/llvm-project-llvmorg/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1327:3 (t+0x6feb4)
#1 __wt_spin_init /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/include/mutex_inline.h:142:5 (libwiredtiger.so.12.0.0+0x31a0d4)
#2 __wti_connection_init /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/conn/conn_handle.c:60:5 (libwiredtiger.so.12.0.0+0x319568)
#3 wiredtiger_open /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/conn/conn_api.c:3120:5 (libwiredtiger.so.12.0.0+0x2dc118)
#4 wts_open /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/test/format/wts.c:841:9 (t+0x143818)
#5 main /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/test/format/t.c:358:9 (t+0x134714)
Mutex M3 (0xffff94204128) created at:
#0 pthread_mutex_init /data/mci/f49c90f0be13703fc74cd438c017c2d0/toolchain-builder/tmp/build-llvm-v5.sh-WmZ/llvm-project-llvmorg/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1327:3 (t+0x6feb4)
#1 __wt_spin_init /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/include/mutex_inline.h:142:5 (libwiredtiger.so.12.0.0+0x501708)
#2 __wt_logmgr_create /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/log/log_mgr.c:1060:5 (libwiredtiger.so.12.0.0+0x501268)
#3 __wti_connection_workers /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/conn/conn_open.c:258:5 (libwiredtiger.so.12.0.0+0x33cc30)
#4 wiredtiger_open /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/conn/conn_api.c:3531:5 (libwiredtiger.so.12.0.0+0x2dff00)
#5 wts_open /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/test/format/wts.c:841:9 (t+0x143818)
#6 main /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/test/format/t.c:358:9 (t+0x134714)
Thread T184 (tid=227882, running) created by main thread at:
#0 pthread_create /data/mci/f49c90f0be13703fc74cd438c017c2d0/toolchain-builder/tmp/build-llvm-v5.sh-WmZ/llvm-project-llvmorg/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1023:3 (t+0x6dfd4)
#1 __wt_thread_create /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/os_posix/os_thread.c:60:5 (libwiredtiger.so.12.0.0+0x54f558)
#2 operations /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/test/format/ops.c:371:9 (t+0x11b130)
#3 main /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/test/format/t.c:393:13 (t+0x134b50)
Thread T47 'log-server' (tid=227370, running) created by main thread at:
#0 pthread_create /data/mci/f49c90f0be13703fc74cd438c017c2d0/toolchain-builder/tmp/build-llvm-v5.sh-WmZ/llvm-project-llvmorg/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1023:3 (t+0x6dfd4)
#1 __wt_thread_create /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/os_posix/os_thread.c:60:5 (libwiredtiger.so.12.0.0+0x54f558)
#2 __wt_logmgr_open /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/log/log_mgr.c:1157:9 (libwiredtiger.so.12.0.0+0x501f74)
#3 __wti_connection_workers /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/conn/conn_open.c:301:5 (libwiredtiger.so.12.0.0+0x33cf30)
#4 wiredtiger_open /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/conn/conn_api.c:3531:5 (libwiredtiger.so.12.0.0+0x2dff00)
#5 wts_open /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/test/format/wts.c:841:9 (t+0x143818)
#6 main /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/test/format/t.c:358:9 (t+0x134714)
format-stress-test-tsan task_log
Logs:
SUMMARY: ThreadSanitizer: data race /data/mci/9350e7cf224f13a0dd02dd2828f62a4f/wiredtiger/src/log/log.c:499:33 in __wt_log_get_backup_files
format-stress-test-tsan task_log
Logs:
#0 0x0000ffff9d06e7b4 in __pthread_kill_implementation () from /lib64/libc.so.6 #0 0x0000ffff9d06e7b4 in __pthread_kill_implementation () from /lib64/libc.so.6 #1 0x0000ffff9d0253a0 [PAC] in raise () from /lib64/libc.so.6 #2 0x0000ffff9d011264 [PAC] in abort () from /lib64/libc.so.6 #3 0x0000aaaad6106204 [PAC] in ___interceptor_abort () at /data/mci/f49c90f0be13703fc74cd438c017c2d0/toolchain-builder/tmp/build-llvm-v5.sh-WmZ/llvm-project-llvmorg/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1926 #4 0x0000aaaad60e429c in Abort () at /data/mci/f49c90f0be13703fc74cd438c017c2d0/toolchain-builder/tmp/build-llvm-v5.sh-WmZ/llvm-project-llvmorg/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp:163 #5 0x0000aaaad60e2184 in __sanitizer::Die() () at /data/mci/f49c90f0be13703fc74cd438c017c2d0/toolchain-builder/tmp/build-llvm-v5.sh-WmZ/llvm-project-llvmorg/compiler-rt/lib/sanitizer_common/sanitizer_termination.cpp:58 #6 0x0000aaaad614e2e0 in finalize () at /data/mci/f49c90f0be13703fc74cd438c017c2d0/toolchain-builder/tmp/build-llvm-v5.sh-WmZ/llvm-project-llvmorg/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2871 #7 0x0000ffff9d027b58 in __run_exit_handlers () from /lib64/libc.so.6 #8 0x0000ffff9d027c44 [PAC] in exit () from /lib64/libc.so.6 #9 0x0000ffff9d01155c [PAC] in __libc_start_call_main () from /lib64/libc.so.6 #10 0x0000ffff9d01163c [PAC] in __libc_start_main_impl () from /lib64/libc.so.6 #11 0x0000aaaad60cd570 [PAC] in _start ()
Repro Artifacts: