Async step-down: rely on transaction write-conflict detection, not the step-down mutex

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Layered Tables
    • None
    • Storage Engines - Foundations
    • 201.301
    • Storage Execution 2026-08-17
    • 13

      Context

      Async step-down lets a disaggregated leader keep serving reads and writes while it hands off leadership, instead of blocking all traffic for the duration. It works off a step-down timestamp set by the application:

      • Everything committed at or before that timestamp belongs to the stable constituent, and to the step-down checkpoint taken at that boundary.
      • Everything committed after it goes to ingest.
      • A transaction is classified once, at begin_transaction, by whether it saw the timestamp set. If it did, its cursors behave like a follower's and write ingest over a still-live stable.
      • Transactions that began before the timestamp was set but commit after it (straddlers) are rolled back.

      Problem

      That split means two concurrent transactions can write the same key into different constituents, one into stable and one into ingest, where WiredTiger's per-tree conflict check cannot see the collision. Today that is handled by:

      • The step-down mutex (txn_global->step_down_lock), a read-write lock: write lock on set and clear, read lock in begin_transaction and commit_transaction.
      • A cross-constituent probe, __clayered_modify_check, which checks both tables for a conflicting update.

      This works, but it puts a corruption-class correctness property on a path of our own making rather than on the engine's existing one.

      Why change it

      • The visibility guarantee rests on a happens-before argument that is reasoned out in the design doc rather than expressed in the code, so every future reader of the step-down timestamp has to re-derive whether it needs the lock.
      • The lock alone is not sufficient: the snapshot is taken outside it, so a stable update can be update-chain-visible without being snapshot-visible. The probe covers that gap, and to do so it treats found-but-invisible as a conflict, which differs from how other lookups in the layered cursor code read the same result.
      • The net effect is two conflict-detection paths with different semantics, and a failure mode that surfaces as silent data corruption rather than an assert.

      Write-conflict detection is long-standing, well-exercised machinery, and the transition window can be made to use it directly.

      Outcome

      Ingest-routed writes double-write to both ingest and stable. Every conflicting pair then meets on the stable btree, so:

      • __wt_txn_modify_check becomes the sole authority on write conflicts.
      • The step-down arm of the cross-table probe goes away.
      • The step-down mutex narrows to the straddler check, which is a plain value check on the timestamp and needs no visibility argument.

      Risk

      Write amplification: every ingest-routed write in the transition window becomes two writes, with the matching cost in cache footprint and reconciliation work on the stable side for as long as the window lasts.

            Assignee:
            Wei Hu
            Reporter:
            Sid Mahajan
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: