Read-only cursor scan can dirty a live stable btree across step-down - "disaggregated btree dirtied while not leader" assertion

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: DHandles
    • None
    • Storage Engines - Transactions
    • 72.011
    • None
    • None

      Problem

      On a leader, a read-only scan can dirty a live stable btree through the deleted-key heuristic: when cursor next/prev leaves a page whose globally visible tombstone count exceeds WT_BTREE_DELETE_THRESHOLD, it calls __wt_page_dirty_and_evict_soon() to force the page through reconciliation (bt_curnext.c / bt_curprev.c).

      The dirty-marking path guards itself with WT_BTREE_READONLY: both __wt_page_modify_set() and __wt_tree_modify_set() return early when the flag is set, and each then asserts the node is the leader. The check and the assert are adjacent relaxed loads, with no lock or generation held across them.

      A scan preempted between them for the duration of a step-down passes the READONLY check as a leader and resumes as a follower: step-down set WT_BTREE_READONLY on the tree and published the follower role in between, and the "disaggregated btree dirtied while not leader" assertion fires (diagnostic builds).

      A pure flag-visibility inversion is not sufficient: step-down's release store of the role is ordered after its READONLY store, so a reader cannot observe the new role together with the old flag. The window is a real preemption spanning an entire step-down.

      Why this is a READONLY issue, not an OUTDATED one

      Step-down sets WT_BTREE_READONLY and WT_DHANDLE_OUTDATED together, so both appear in any trace of the race, but only READONLY participates:

      • the dirty-marking path reads READONLY and the role, and never reads OUTDATED;
      • OUTDATED governs only the aftermath — it is a required conjunct of __wt_btree_is_outdated_disagg(), which arms the eviction gates that discard the wrongly dirtied page instead of reconciling it.

      Consequently, fixing the OUTDATED update (WT-18452, WT-18334) does not close this hole, and closing this hole does not remove the need for those gates: pages dirtied legitimately during the leader era still require the discard routing.

      Impact

      • Diagnostic builds: assertion abort at the dirty-marking site.
      • Production builds: the page is dirtied silently on a follower. The content change is a no-op — removal of globally visible tombstones — and the dirty bit is only an eviction accelerator. Disposal is safe on both normal paths: the eviction review gate clears the dirty bit and discards the page, and sweep discards the outdated handle without reconciling it.
      • If those gates are bypassed (for example the flag update is lost — WT-18452 / WT-18334), reconciliation of a READONLY tree panics in every build: WT_ASSERT_ALWAYS, "Attempting reconciliation on a read-only page" (rec_write.c). The block-manager follower-write assertion is not reachable for this tree, because READONLY is set well before eviction runs.
      • No silent corruption in any build.

      Reproduced

      Deterministic reproducer in a comment below (temporary, not for commit): an env-armed sleep hook in __wt_tree_modify_set between the READONLY check and the leader assert. A scan of a delete-heavy live stable btree suspends there, a full step-down completes during the sleep, and the assertion fires on resume (3/3 runs, SIGABRT). The control run with the hook unarmed passes.

            Assignee:
            Shoufu Du
            Reporter:
            Ivan Kochin
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: