Major - P3 The data race reported by TSAN lies in unsynchronized accesses to `
dhandle->timeofdeath` that marks the dhandle as a candidate for sweeping.
Based on my understanding of the code that should be fine to make all the accesses relaxed since all the further actions that happens after reading non-zero `timeofdeatch` happens under locking or affects only session local cache, but I am curious about synchronization across the dhandles and dhandle queue since in some places it's done under the lock, and it some places it's done without any synchronization, so I think that could be important to understand the way of dhandles synchronization.
One particular case I am curious about, is whether it is dangerous if one sweep_thread checks and see that `now - dhandle->timeofdeath > conn->sweep_idle_time` so it starts closing the dhandle by acquiring WRITE lock, but I am not sure whether other threads that accesses dhandle hold read lock (I think they should).
Anyway, I decided to create this ticket to continue investigation later.
TSAN warning:
==================WARNING: ThreadSanitizer: data race (pid=14915) Read of size 8 at 0xffffb43884c8 by main thread (mutexes: write M0): #0 __wt_cursor_dhandle_incr_use /home/ec2-user/work/git/wiredtiger-arm/src/include/cursor_inline.h:381 (libwiredtiger.so.12.0.0+0x191ba4) #1 __curfile_create /home/ec2-user/work/git/wiredtiger-arm/src/cursor/cur_file.c:1082 (libwiredtiger.so.12.0.0+0x191ba4) #2 __wt_curfile_open /home/ec2-user/work/git/wiredtiger-arm/src/cursor/cur_file.c:1285 (libwiredtiger.so.12.0.0+0x191ba4) #3 __session_open_cursor_int /home/ec2-user/work/git/wiredtiger-arm/src/session/session_api.c:697 (libwiredtiger.so.12.0.0+0x2d8d78) #4 __wt_open_cursor /home/ec2-user/work/git/wiredtiger-arm/src/session/session_api.c:795 (libwiredtiger.so.12.0.0+0x2d8a58) #5 __wt_curtable_open /home/ec2-user/work/git/wiredtiger-arm/src/cursor/cur_table.c:999 (libwiredtiger.so.12.0.0+0x1ef10c) #6 __session_open_cursor_int /home/ec2-user/work/git/wiredtiger-arm/src/session/session_api.c:656 (libwiredtiger.so.12.0.0+0x2d9034) #7 __session_open_cursor /home/ec2-user/work/git/wiredtiger-arm/src/session/session_api.c:855 (libwiredtiger.so.12.0.0+0x2daae0) #8 add_work /home/ec2-user/work/git/wiredtiger-arm/examples/c/ex_backup.c:140 (ex_backup+0xf793c) #9 main /home/ec2-user/work/git/wiredtiger-arm/examples/c/ex_backup.c:449 (ex_backup+0xf6780) Previous write of size 8 at 0xffffb43884c8 by thread T13: #0 __sweep_mark /home/ec2-user/work/git/wiredtiger-arm/src/conn/conn_sweep.c:109 (libwiredtiger.so.12.0.0+0x16c144) #1 __sweep_server /home/ec2-user/work/git/wiredtiger-arm/src/conn/conn_sweep.c:461 (libwiredtiger.so.12.0.0+0x16c144) Location is heap block of size 512 at 0xffffb4388400 allocated by main thread: #0 calloc /data/mci/0701f0b853b8865c4e2f06b43ba791fc/toolchain-builder/tmp/build-llvm-v5.sh-u2l/llvm-project-llvmorg/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:686 (ex_backup+0x67d24) #1 __wt_calloc /home/ec2-user/work/git/wiredtiger-arm/src/os_common/os_alloc.c:68 (libwiredtiger.so.12.0.0+0x26a158) #2 __wt_conn_dhandle_alloc /home/ec2-user/work/git/wiredtiger-arm/src/conn/conn_dhandle.c:210 (libwiredtiger.so.12.0.0+0x155208) #3 __session_find_shared_dhandle /home/ec2-user/work/git/wiredtiger-arm/src/session/session_dhandle.c:848 (libwiredtiger.so.12.0.0+0x301020) #4 __session_get_dhandle /home/ec2-user/work/git/wiredtiger-arm/src/session/session_dhandle.c:878 (libwiredtiger.so.12.0.0+0x301020) #5 __wt_session_get_dhandle /home/ec2-user/work/git/wiredtiger-arm/src/session/session_dhandle.c:908 (libwiredtiger.so.12.0.0+0x300174) #6 __create_file /home/ec2-user/work/git/wiredtiger-arm/src/schema/schema_create.c:307 (libwiredtiger.so.12.0.0+0x2c4204) #7 __schema_create /home/ec2-user/work/git/wiredtiger-arm/src/schema/schema_create.c:1520 (libwiredtiger.so.12.0.0+0x2c2654) #8 __wt_schema_create /home/ec2-user/work/git/wiredtiger-arm/src/schema/schema_create.c:1580 (libwiredtiger.so.12.0.0+0x2c2654) #9 __create_colgroup /home/ec2-user/work/git/wiredtiger-arm/src/schema/schema_create.c:620 (libwiredtiger.so.12.0.0+0x2c36c8) #10 __create_table /home/ec2-user/work/git/wiredtiger-arm/src/schema/schema_create.c:? (libwiredtiger.so.12.0.0+0x2c61c0) #11 __schema_create /home/ec2-user/work/git/wiredtiger-arm/src/schema/schema_create.c:1528 (libwiredtiger.so.12.0.0+0x2c276c) #12 __wt_schema_create /home/ec2-user/work/git/wiredtiger-arm/src/schema/schema_create.c:1580 (libwiredtiger.so.12.0.0+0x2c276c) #13 __wt_session_create /home/ec2-user/work/git/wiredtiger-arm/src/session/session_api.c:1032 (libwiredtiger.so.12.0.0+0x2d92f0) #14 __session_create /home/ec2-user/work/git/wiredtiger-arm/src/session/session_api.c:1074 (libwiredtiger.so.12.0.0+0x2df358) #15 main /home/ec2-user/work/git/wiredtiger-arm/examples/c/ex_backup.c:435 (ex_backup+0xf6608) Mutex M0 (0xffffb71463b0) created at: #0 pthread_mutex_init /data/mci/0701f0b853b8865c4e2f06b43ba791fc/toolchain-builder/tmp/build-llvm-v5.sh-u2l/llvm-project-llvmorg/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1327 (ex_backup+0x6b934) #1 __wt_spin_init /home/ec2-user/work/git/wiredtiger-arm/src/include/mutex_inline.h:142 (libwiredtiger.so.12.0.0+0x2db3c4) #2 __open_session /home/ec2-user/work/git/wiredtiger-arm/src/session/session_api.c:2428 (libwiredtiger.so.12.0.0+0x2db3c4) #3 __wt_open_session /home/ec2-user/work/git/wiredtiger-arm/src/session/session_api.c:2551 (libwiredtiger.so.12.0.0+0x2db3c4) #4 __conn_open_session /home/ec2-user/work/git/wiredtiger-arm/src/conn/conn_api.c:1379 (libwiredtiger.so.12.0.0+0x141df0) #5 main /home/ec2-user/work/git/wiredtiger-arm/examples/c/ex_backup.c:434 (ex_backup+0xf65cc) Thread T13 'sweep-server' (tid=14931, running) created by main thread at: #0 pthread_create /data/mci/0701f0b853b8865c4e2f06b43ba791fc/toolchain-builder/tmp/build-llvm-v5.sh-u2l/llvm-project-llvmorg/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1023 (ex_backup+0x69a54) #1 __wt_thread_create /home/ec2-user/work/git/wiredtiger-arm/src/os_posix/os_thread.c:60 (libwiredtiger.so.12.0.0+0x277d04) #2 __wti_sweep_create /home/ec2-user/work/git/wiredtiger-arm/src/conn/conn_sweep.c:553 (libwiredtiger.so.12.0.0+0x16b8d0) #3 __wti_connection_workers /home/ec2-user/work/git/wiredtiger-arm/src/conn/conn_open.c:302 (libwiredtiger.so.12.0.0+0x1640a0) #4 wiredtiger_open /home/ec2-user/work/git/wiredtiger-arm/src/conn/conn_api.c:3324 (libwiredtiger.so.12.0.0+0x13d5ec) #5 main /home/ec2-user/work/git/wiredtiger-arm/examples/c/ex_backup.c:431 (ex_backup+0xf6508) SUMMARY: ThreadSanitizer: data race /home/ec2-user/work/git/wiredtiger-arm/src/include/cursor_inline.h:381 in __wt_cursor_dhandle_incr_use==================ThreadSanitizer: reported 1 warnings