-
Type:
Task
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: Checkpoint Cleanup
-
Storage Engines - Transactions
-
23.767
-
SE Transactions - 2026-09-11
-
3
Background
Design doc: "Improve Checkpoint Cleanup in Disagg" (Chenhao Qu) — https://docs.google.com/document/d/1f84mvZElLOj2FqSXujdIaYsBs0bV-oyFLLBcjloXE4w/edit
Checkpoint cleanup reclaims obsolete data in two ways:
- Page-level GC: a fully-obsolete page is marked deleted in its parent without reading it into cache. This is fine for disaggregated storage — the entire delta chain and base image can be released together.
- Key-level GC: a page with only some obsolete keys/time windows is read into cache and marked dirty so reconciliation rewrites it without the obsolete data.
In disaggregated storage, reconciliation produces deltas rather than full pages. Marking a page dirty for key-level cleanup is counterproductive there:
- A globally-visible deletion still has to be recorded as a tombstone in the new delta, so cleanup adds data instead of reclaiming it.
- A globally-visible start time point that hasn't otherwise changed is simply omitted from the delta (no savings), or if written anyway, adds data for no benefit.
This also adds unnecessary write amplification against SLS (higher bandwidth/cost) with no compactor running during private preview to offset it.
Current state (confirmed against develop)
Key-level GC is not currently disabled for disaggregated trees. The only disagg-aware gating in checkpoint cleanup is thread-level (follower skips running the cleanup thread entirely, leader-only scheduling — WT-15535, WT-18142). The obsolete-time-window path that marks a page dirty for key-level cleanup has no WT_BTREE_DISAGGREGATED guard:
- src/btree/bt_sync_obsolete.c, _sync_obsolete_inmem_evict_or_mark_dirty (obsolete time-window branch calls _wt_page_modify_set unconditionally).
- src/btree/bt_sync_obsolete.c, _checkpoint_cleanup_page_skip overrides its skip decision via _sync_obsolete_tw_check to force these pages to be read, unconditionally for disagg trees too.
- __checkpoint_cleanup_eligibility excludes tiered .wtobj tables and read-only btrees, but disagg file: tables remain eligible for the key-level path.
Page-level fast-delete GC (_sync_obsolete_disk_cleanup / _sync_obsolete_deleted_cleanup) is unaffected and should keep running for disagg trees.
Task (Stage 1 of the design doc)
Add a disagg guard so checkpoint cleanup skips the key-level (mark-dirty-for-obsolete-time-window) path for disaggregated btrees, while leaving page-level GC intact. Likely touch points: the obsolete time-window branch in _sync_obsolete_inmem_evict_or_mark_dirty and the forced-read override in checkpoint_cleanup_page_skip/_sync_obsolete_tw_check in src/btree/bt_sync_obsolete.c.
This is a nice-to-have before private preview, not a blocker. Stage 2 (a real cost/benefit algorithm to re-enable key-level GC selectively) and Stage 3 (tuning) are tracked separately per the design doc.
Estimate: ~1 engineering week (Persistence team).