ExportXMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Cache and Eviction
    • None
    • Storage Engines - Transactions
    • 96.08
    • SE Transactions - 2026-12-18
    • None

      Problem

      Checkpoint cleanup reads every on-disk internal page whose subtree has a timestamped stop (newest_stop_durable_ts != WT_TS_NONE), on every pass (checkpoint_cleanup.wait, default 300 s), whatever checkpoint_cleanup.method is. It skips the on-disk leaf children, so each page it reads stays in cache with no resident child.

      When the cache is in no-keep mode, those reads get WT_READGEN_WONT_NEED. Cleanup walks with WT_READ_NO_EVICT, so on release each page goes to the urgent queue:

      __checkpoint_cleanup_walk_btree
      └─ __wt_tree_walk_custom_skip(..., __checkpoint_cleanup_page_skip, WT_READ_NO_EVICT | WT_READ_VISIBLE_ALL)
          ├─ __checkpoint_cleanup_page_skip → read: on-disk internal page with a timestamped stop below it
          └─ __wt_page_release(..., WT_READ_NO_EVICT)
              └─ __wt_page_evict_soon_check() true (WONT_NEED)
                  └─ __wt_page_evict_urgent()   ← wakes the eviction workers on every push
      

      Cleanup only stops reading above eviction_trigger, so this runs throughout the no-keep range.

      Observed (HELP-103000, 8.0.37, large collection with many range deletes)

      • Cleanup passes run every 300 s for about 45-60 s, visiting about 150-190M child refs per pass on the large collection.
      • At rest (no queries on the collection): the collection's ~0.9M internal-page reads in 36 min are all cleanup; internal pages are 99.9% of its evictions.
      • At 85%+ fill (no-keep), during a pass vs the 120 s before it (medians, 2 nodes, 56 passes):
        • urgent queue pushes: ~1k/s → 14-15k/s; internal pages evicted: ~1.3k/s → 14-15k/s
        • get-page calls: 7-9k/s → 57-61k/s; "found queue empty": 28-30% → 67-68%
        • ordinary walk unchanged: pages seen, pages queued and queued/seen flat or slightly up
      • So each pass reads ~14k internal pages/s that are evicted almost at once and free little space, and the urgent wakeups inflate the get-page queue-empty statistic.

      What to investigate

      1. Confirm the loop locally: large tree with timestamped deletes, cache held in no-keep, cleanup on vs. a long wait; measure urgent pushes, worker time, read volume and fill.
      2. Decide what cleanup should do under cache pressure, for example: do not read internal pages while no-keep is active (or above eviction_target), or do not urgent-queue pages released by the cleanup walk.
      3. Check whether the queue-empty statistic should exclude wakeups caused by urgent pushes.

      Definition of done

      A reproducer showing the read-then-urgent-evict loop, and either a change that removes it without reducing cleanup's obsolete-page removal, or a documented reason to keep it.

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Shoufu Du
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: