Allow the cursor walk to skip clean in-memory internal pages

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Btree
    • Storage Engines, Storage Engines - Transactions
    • 0.069
    • SE Transactions - 2026-09-11
    • 3

      Summary

      __wt_btcur_skip_page() refuses to consider any internal page that is not in WT_REF_DISK:

      if (F_ISSET(ref, WT_REF_FLAG_INTERNAL) && previous_state != WT_REF_DISK)
          goto unlock;
      

      The reasoning is that an in-memory internal page cannot be judged by the aggregate in its address cell, because a descendant may be dirty with newer data than the aggregate reports and reconciliation is what propagates that upwards.

      That reasoning does not hold for a clean in-memory internal page. Such a page carries no unreconciled data, so its address cell aggregate is as trustworthy as that of a page still on disk. It can be treated the same way an on-disk internal page already is, and become eligible for the walk skip. The leaf path already makes exactly this distinction through its clean_page handling; the internal path returns before reaching it.

      The reviewer of PR #14629 asked for a separate ticket to skip clean in-memory internal pages in the tree walk.

      Approach

      • In __wt_btcur_skip_page() (src/include/btree_inline.h), stop returning early for an internal page in WT_REF_MEM that is not modified; let it fall through to the same address-cell evaluation used for WT_REF_DISK, reusing the existing clean_page determination.
      • Establish the correctness condition precisely: the address cell aggregate of a clean internal page describes its state as of the last reconciliation, and a dirty descendant marks its ancestors dirty through __wt_page_modify_set(). Verify this holds on every path that can leave an ancestor clean above modified content, including split and instantiated-delete paths.
      • Keep the existing state lock discipline: the decision is made under WT_REF_TRYLOCK so checkpoint and eviction cannot change the aggregate underneath.
      • Account the skip through __wt_btcur_skip_page_inc() so the new case is visible in the walk-skip statistics.

      Risk

      Treating a clean internal page as on-disk is wrong if any descendant holds newer data than the aggregate reports; the walk would then silently hide live records. The visibility argument must be airtight before this lands, and the change should be exercised under format and the fast-truncate suites with eviction and checkpoint active.

      Definition of done

      • A clean in-memory internal page is evaluated through its address cell and skipped on the same terms as an on-disk internal page.
      • A dirty internal page is still descended into.
      • Test coverage that fails if the clean in-memory internal skip is applied where the subtree still holds visible records.

            Assignee:
            Shoufu Du
            Reporter:
            Shoufu Du
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: