Use atomic operations for btree->modified

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Btree
    • None
    • Storage Engines - Transactions
    • 87.583
    • None
    • None

      WT_BTREE.modified is declared wt_shared bool modified (src/include/btree.h:228), but every one of its ~22 read and write sites is a plain access – none go through _wt_atomic*. The wt_shared annotation marks the field as concurrently accessed without making the accesses atomic, so this is a formal data race.

      The accesses are genuinely cross-thread

      This is not a theoretical pairing; the code documents it. The hot dirtying path in __wt_page_modify_set (src/include/btree_inline.h:1122-1142) does a plain test-then-set guarded by a hand-rolled barrier, and its own comment states the sharing:

      Test before setting the dirty flag, it's a hot cache line.

      The tree's modified flag is cleared by the checkpoint thread: set it and insert a barrier before dirtying the page.

      if (!btree->modified) {
          ...
          btree->modified = true;
          WT_FULL_BARRIER();
      

      Writers include tree open (bt_handle.c:804, :894), checkpoint (checkpoint_txn.c:1091, :3335, :3474), reconciliation (rec_write.c:627) and the dirtying path above. Readers include the sweep server (conn_sweep.c:158), checkpoint (bt_sync.c:330, :333, :596; checkpoint_txn.c:2778, :2983, :3680, :3794, :3801), RTS (rts_history.c:176, rts_btree_walk.c:21) and the cursor walk (bt_page.c:1238). No single lock covers all of these paths.

      It is already papered over in one place

      checkpoint_txn.c:3335 cannot be written plainly without tripping TSan, so it uses __wt_tsan_suppress_store_bool(&btree->modified, false). src/include/tsan_suppress.h describes those wrappers as a deliberately temporary measure:

      we still add these functions to the suppression file to ensure we don't forget to properly fix them later and to highlight that this is only a temporary solution while we investigate high-priority TSAN warnings.

      So the field already has one acknowledged-unfixed race; converting it removes the suppression rather than adding another.

      Why it is worth doing

      • Removes a formal data race on a field read by four different background subsystems.
      • Replaces WT_FULL_BARRIER() on a hot cache line with a release store, which is both cheaper and states the intended ordering explicitly instead of leaving it to a comment.
      • Lets the TSan suppression at checkpoint_txn.c:3335 and its suppression-file entry be deleted.
      • WT-18427 added a read of this field on the drop path, to decide whether an already-clean tree can be marked dead and have its cache discard deferred to sweep. That read is correct today because it quiesces eviction before trusting a clean result, but it is still a plain read of a concurrently written field.

      Deliverables

      • Convert all reads and writes to _wt_atomic* with an explicit memory order per site, rather than the blanket-sequential-consistency or hand-rolled-barrier approaches.
      • Pay particular attention to the test-then-set in __wt_page_modify_set: decide whether it wants a relaxed load plus a release store, and whether the surrounding WT_FULL_BARRIER() can then be removed.
      • Determine the ordering each reader actually needs – the sweep and checkpoint readers use the value to decide whether a tree is safe to close or skip, so an acquire may be required, whereas the statistics-style reads can stay relaxed.
      • Remove the __wt_tsan_suppress_store_bool call at checkpoint_txn.c:3335 and its suppression-file entry.

      Scope note

      WT-18015 audited the WT_BTREE flags bitfield and moved concurrently mutated flags to flags_atomic. modified is a separate standalone field and was outside that ticket's scope, so it was not converted then.

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Chenhao Qu
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: