-
Type:
Task
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: Btree
-
Storage Engines - Transactions
-
541.649
-
SE Transactions - 2026-08-28, SE Transactions - 2026-09-11
-
5
Summary
A WT_REF's address cell on the parent carries a WT_TIME_AGGREGATE, and that aggregate is recursive: internal-page reconciliation copies each child ref's aggregate into the cell it writes (__rec_cell_build_addr in rec_row.c / rec_col.c), and rec_write.c merges every written cell into the page's own aggregate. So an internal ref's aggregate is a merge over its entire subtree, not just its direct children.
That means we can already tell, without reading an internal page, whether every record beneath it has been deleted and whether those deletes are visible:
- all deleted: WT_TIME_AGGREGATE_ALL_DELETED() – i.e. newest_stop_ts != WT_TS_MAX, since any surviving record leaves WT_TS_MAX in the merge.
- visible: the same snapshot check the leaf path already uses, _wt_txn_snap_min_visible() on the aggregated stop point (or _wt_txn_visible_all() after WT_TIME_AGGREGATE_MERGE_OBSOLETE_VISIBLE() where global visibility is wanted).
Both are reachable from the parent via __wt_ref_addr_copy().
Proposal
Use this to skip the read (and the walk) of a WT_REF_DISK internal ref whose subtree is entirely deleted and visible. The intended scope is read-only skipping – tree walks, cursor traversal, verify, statistics – where the win is avoiding I/O and cache pressure for a subtree that can produce no visible content.
Primary implementation site: __wt_btcur_skip_page()
__wt_btcur_skip_page() in src/include/btree_inline.h already performs exactly this check for leaf pages – it takes the ref lock, consults fast-truncate state and then the address cell's WT_TIME_AGGREGATE, and skips on WT_TIME_AGGREGATE_HAS_STOP() && !ta.prepare && __wt_txn_snap_min_visible(...). It explicitly declines to do so for internal pages:
Skip this test on an internal page, as we rely on reconciliation to mark the internal page dirty. There could be a period of time when the internal page is marked clean but the leaf page is dirty and has newer data than let on by the internal page's aggregated information.
if (F_ISSET(ref, WT_REF_FLAG_INTERNAL)) return (0);
This is the crux of the ticket, and the stated hazard is the real obstacle rather than the absence of information. The aggregate on an internal ref is only as current as the last reconciliation that wrote it; a descendant leaf can be dirty with newer content while the internal page above it is still clean, so the aggregate can claim "all deleted" for a subtree that is no longer entirely deleted in memory.
Directions to evaluate:
- Restrict the internal-page skip to refs in WT_REF_DISK state. A subtree that is wholly on disk has no in-memory descendants that could contradict the aggregate. This needs confirming against the eviction and checkpoint paths – in particular whether a WT_REF_DISK internal ref can be reached while a descendant is being instantiated.
- Alternatively, find or introduce a marker that lets a reader tell that no descendant has been modified since the internal page was last reconciled, so the clean-parent/dirty-child window is detectable rather than merely improbable.
The existing ref-locking and backoff in __wt_btcur_skip_page() carries over unchanged; extend WT_PAGE_WALK_SKIP_STATS with a counter for skipped subtrees so the optimisation is measurable.
Explicitly out of scope
Removing or truncating such a subtree. The two consumers that act on the aggregate destructively both gate on addr.type == WT_ADDR_LEAF_NO on purpose:
- fast-truncate, __wt_delete_page() in bt_delete.c
- checkpoint-cleanup obsolete page removal, __sync_ref_obsolete_check() in bt_sync_obsolete.c
The blockers for extending those to internal refs:
- Block reclamation. Discarding a leaf ref frees one extent the parent already knows about. Discarding an internal ref orphans every block in the subtree, and the extents cannot be returned to the block manager without reading the pages to find them – which defeats the point. WT_ADDR_LEAF_NO encodes the same constraint for overflow blocks.
- Truncate bookkeeping. page_del and instantiation-on-read are per-leaf-ref concepts; there is no home for deletion state on the intermediate refs of a skipped subtree.
Other correctness constraints
- ta.prepare must still be honoured; it does propagate through the merge.
- Handle a missing or empty aggregate (__wt_ref_addr_copy() returning false, WT_TIME_AGGREGATE_IS_EMPTY()) conservatively by not skipping.
Suggested work
- Settle the clean-parent/dirty-child question above; it determines whether the optimisation is safe at all.
- Relax the internal-page early return in __wt_btcur_skip_page() under whatever condition that analysis produces.
- Add a statistic for skipped subtrees, and a test that builds a multi-level tree, deletes and makes visible an entire subtree, checkpoints, reopens, and asserts the internal pages are never read in.
- is depended on by
-
WT-18496 Reconciliation should merge the effective (page_del-applied) aggregate for WT_CELL_ADDR_DEL children into the parent, not the stale raw child aggregate
-
- Closed
-
- is related to
-
WT-18407 Prioritise eviction of internal pages whose children are fast-truncated
-
- Open
-
-
WT-18536 Reclaim a skipped disaggregated subtree once its truncate is globally visible
-
- Closed
-
-
WT-18550 Checkpoint cleanup dirties a fully deleted disaggregated internal page for a no-progress rewrite before its truncate is globally visible
-
- Closed
-
-
WT-18524 Evaluate skipping emptied internal pages on non-cursor walk and descent paths
-
- Needs Scheduling
-
-
WT-18525 Evaluate skipping emptied internal pages for the local block manager
-
- Needs Scheduling
-
- related to
-
WT-18496 Reconciliation should merge the effective (page_del-applied) aggregate for WT_CELL_ADDR_DEL children into the parent, not the stale raw child aggregate
-
- Closed
-
-
WT-18536 Reclaim a skipped disaggregated subtree once its truncate is globally visible
-
- Closed
-
-
WT-18550 Checkpoint cleanup dirties a fully deleted disaggregated internal page for a no-progress rewrite before its truncate is globally visible
-
- Closed
-
-
WT-18524 Evaluate skipping emptied internal pages on non-cursor walk and descent paths
-
- Needs Scheduling
-
-
WT-18525 Evaluate skipping emptied internal pages for the local block manager
-
- Needs Scheduling
-