Non-transactional ingest table clear can trigger WT_ROLLBACK and panic step-up

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • WT12.0.0, 9.1.0-rc0
    • Affects Version/s: None
    • Component/s: Truncate
    • None
    • Storage Engines - Transactions
    • 355.925
    • SE Transactions - 2026-08-28, SE Transactions - 2026-09-11
    • 5

      During disagg step-up on the new primary, a drain worker failed to clear an ingest table and the WT_THREAD_PANIC_FAIL thread group turned it into a panic:

      2026-08-27T19:17:22.636 [WT] __layered_drain_worker_run:856: Failed to clear ingest table
          "file:collection-96883c24-fca0-4443-93ed-f1eb93f2fee7.wt_ingest":
          WT_ROLLBACK: conflict between concurrent operations
      [WT] __thread_run:47: Unrecoverable utility thread error
      [WT] __wt_panic ... -> mongod fassert 50853 (wiredtiger_util.cpp) -> node aborted
      

      Initial investigation shows a bug of self-inflicted sequence when truncate the ingest table, no concurrent writer required:

      1. Tombstone key K (globally visible), call next().
      2. next() returns WT_RESTART — e.g. an in-memory split (the tombstones themselves grow page footprint), or the page being evicted/pruned under the cursor (the ingest btree is WT_BTREE_GARBAGE_COLLECT; the clear's own globally-visible tombstones make pages prunable, and the drain typically runs with the cache heavily dirty).
      3. Retry re-searches K, which is now invisible (its own tombstone) or physically pruned, so search returns WT_NOTFOUND.
      4. The final mapping converts that to WT_ROLLBACK; the drain worker error panics the process.

      The NOTFOUND-to-ROLLBACK mapping is correct for transactional truncates (start key concurrently removed = retryable conflict), but for the non-transactional clear the truncate's own globally-visible writes can trigger it.

      This ticket is to verify this bug, propose a solution and fix it.

            Assignee:
            Zunyi Liu
            Reporter:
            Zunyi Liu
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: