Fix test_disagg_fast_truncate03 flaky assertion from walk-skip event recounting

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • WT12.0.0
    • Affects Version/s: None
    • Component/s: Test Python
    • None

      test_disagg_fast_truncate03 flaked once on the amazon2023 ARM64 spinlock variant (wiredtiger_amazon2023_arm64_spinlock_pthread_adaptive_test, patch build of 2026-09-07):

      AssertionError: 10 not greater than or equal to 18 : step 9: a skipped internal page did not reconcile to empty

      ROOT CAUSE

      The step-9 assertion compares rec_page_delete (pages reconciled to empty) against the cursor_tree_walk_del_internal_page_skip value measured during the step-5 scan. That statistic counts page examination events, not distinct pages: when the cursor's tree walk restarts (WT_RESTART after a concurrent page split, e.g. an eviction-triggered in-memory split), the walk re-descends from its starting position and re-examines - and recounts - internal pages it already skipped. The measured value can therefore exceed the true number of skippable pages (observed 107 events for 58 distinct pages). Reclamation itself is complete and correct: per-page event tracing shows every checkpoint-cleanup-processed internal page reconciles to empty in the same checkpoint pass, and the leaf block discards all happen. So this is a test assertion bug, not a storage-engine bug.

      REPRODUCTION

      Needs a wide timing window: spinlock build (SPINLOCK_TYPE=pthread_adaptive) on ARM64 Linux, a deeper tree, and an oversubscribed machine; the new checkpoint_cleanup_slow timing stress widens the cleanup-to-checkpoint window further. Without the fix: ~1.3% failure rate in that configuration. With the fix: 0 failures in ~19k runs.

      FIX

      • Step 9 no longer compares against the event count; it checkpoints until rec_page_delete converges (reclamation driven to completion, deadline-bounded), and keeps the block-discard assertion.
      • The exit step additionally asserts that a final scan observes no more internal-page skips and no read-backs - the true end-state property the test wants.
      • Adds the checkpoint_cleanup_slow timing-stress flag (used while reproducing).

      FOLLOW-UP (not fixed here): a rarer step-5 signature ("the evicted internal page was read back", seen 2x in ~15k stressed runs on bare-metal ARM64) likely belongs to the same walk-restart family but is unpinned as of filing.

            Assignee:
            Shoufu Du
            Reporter:
            Shoufu Du
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: