ExportXMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Checkpoints
    • None
    • Storage Engines - Persistence
    • 295.272
    • StorEng - Defined Pipeline
    • None

      WTI_CKPT_THREAD::signalled (src/checkpoint/checkpoint_private.h:82) is a plain bool that two threads access without a common lock:

      • the checkpoint server clears it in __ckpt_server (src/checkpoint/checkpoint_conn.c:110) after a log-size checkpoint, holding no lock;
      • the log server reads it, and sets it, in _wt_checkpoint_signal (src/checkpoint/checkpoint_conn.c:244 and :246), called from _wti_log_release (src/log/log.c:1928) under the log slot lock.

      TSan reports this as a data race. The race is only reachable when checkpoints are triggered by the amount of log written (checkpoint=(log_size=...)).

      Seen in format-stress-test-tsan on amazon2023-arm64-tsan, on a PR patch that does not touch checkpoint or logging code: patch task. The format run itself succeeded; the process aborted at exit on the TSan report (abort_on_error=1). The relevant run parameters were logging=1, checkpoint=wiredtiger, checkpoint.log_size=36 and disagg.mode=off.

      TSan report (abridged)
      WARNING: ThreadSanitizer: data race (pid=18784)
        Write of size 1 at 0xffffb2f0b368 by thread T86:
          #0 __ckpt_server src/checkpoint/checkpoint_conn.c:110:41
        Previous read of size 1 at 0xffffb2f0b368 by thread T66 (mutexes: write M0):
          #0 __wt_checkpoint_signal src/checkpoint/checkpoint_conn.c:244:26
          #1 __wti_log_release src/log/log.c:1928:9
          #2 __log_slot_switch_internal src/log/log_slot.c:382:9
          #3 __wti_log_slot_switch src/log/log_slot.c:415:9
          #4 __wti_log_force_write src/log/log.c:2510:13
          #5 __log_server src/log/log_mgr.c:982:13
        Mutex M0 created at:
          #1 __wt_spin_init src/include/mutex_inline.h:142:5
          #2 __wt_logmgr_create src/log/log_mgr.c:1070:5   (log_slot_lock)
        Thread T86 'checkpoint-serv'
        Thread T66 'log-server'
      SUMMARY: ThreadSanitizer: data race src/checkpoint/checkpoint_conn.c:110:41 in __ckpt_server
      

      The race is old. The flag has been accessed this way since WT-13913 (f3f3b0cb89) moved it into WTI_CKPT_THREAD, and before that in src/conn/conn_ckpt.c. WT-15799 (cec2d68c2a) made the neighbouring logsize field wt_shared with atomic loads but left signalled plain. It is rare: the two format-stress-test-tsan failures on this variant in the last ~93 mainline runs are both WT-18861, a different race.

      Likely impact (not verified): at worst a lost or duplicated log-size checkpoint signal, so one checkpoint starts late or an extra one runs; no data is at risk.

      Suggested fix: declare signalled as wt_shared and use _wt_atomic_load_bool_relaxed / _wt_atomic_store_bool_relaxed at checkpoint_conn.c:110, :244 and :246.

      Related: WT-16488 (fixed a different close race in the same signalling path, 5dc14c089b).

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Alex Blekhman
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: