Found while root-causing AF-21528 from a core dump (Kudzu release-7-2, build 85479c1d08a2).

Summary

On a disaggregated btree marked outdated at step-down, eviction force-clears a page that has already been reconciled. That routes the page down the clean-eviction path, which caches page->dsk in the victim cache. But page->dsk is the prereconciliation image, while page>disagg_info->block_meta has already been advanced to the post-reconciliation identity. The result is stale content published under a newer LSN/delta identity, which later readers accept without any verification.

Mechanism

1. __wt_page_evict_clean() is page->modify == NULL || (!modified && rec_result == 0) (btree_inline.h:124). The outdated-disagg gate at evict_page.c:477-482 therefore fires only for pages that do have a reconciliation result:

if (__wt_btree_is_outdated_disagg(session) && !__wt_page_evict_clean(page)) {
    if (__wt_atomic_load_int32_relaxed(&session->dhandle->session_inuse) > 0) {
        ret = __wt_set_return(session, EBUSY);
        goto err;
    }
    __wt_page_modify_clear(session, page);
}

2. __wt_page_modify_clear() (btree_inline.h:1174) sets page_state = WT_PAGE_CLEAN and modify->flags = 0. It does not clear modify->rec_result.

3. At evict_page.c:490 is_dirty stays false, so the dispatch at evict_page.c:560 takes _evict_page_clean_update() instead of _evict_page_dirty_update(). mod->mod_replace — the newly written image — is never honoured.

4. _evict_page_clean_update() calls _evict_page_victim_cache() (evict_page.c:681).

5. The eligibility check (evict_page.c:113-115) is commented "Only cache clean pages without modify" but tests only __wt_page_is_modified(page), which step 2 just made false. The page passes a guard that was meant to exclude it.

6. The put caches .data = page->dsk while tagging it with .lsn = block_meta.disagg_lsn, .base_lsn, .delta_count (evict_page.c:169-175, 251-260). Reconciliation advanced page->disagg_info->block_meta at rec_write.c:3353 but never rewrote page->dsk, so the two describe different versions.

Evidence from the AF-21528 core

The crashing read was served from the stable side of a layered cursor bound to the latest checkpoint (cursor stable_checkpoint_meta_lsn == conn last_checkpoint_meta_lsn == 7684043182304134635, pending equal, deferred queue empty), so this is not a stale-checkpoint or missed-pickup problem.

The page it read:

page_id        = 14606
disagg_lsn     = 7684041889518977194  (22:46:59.170)
base_lsn       = 7684040665453297937  (22:42:14.273)
delta_count    = 1
backlink_lsn   = 0
shared_dsk_item non-NULL; page->dsk == shared_dsk_item->data

The address cookie in the parent internal page is byte-identical to the shared-disk-cache key, and every decoded field (page_id, lsn, base_lsn, size, checksum) matches the cached block_meta exactly — yet the document bytes in the image are the 22:42:13 version, i.e. the base image, with the 22:46:59 delta not applied.

Walking all 100 shared-disk-cache entries: reconstructed-from-deltas images carry a zeroed disagg header, plain base images carry magic=0xdb. Only four entries claim delta_count > 0 while carrying a base-magic image with WT_BLOCK_DISAGG_MODIFIED — the flag stamped only by the victim-cache put at evict_page.c:238:

page_id  fid  dc  disagg_lsn ts        base_lsn ts          magic flags
14281    601  6   1789080419=22:46:59  1789079845=22:37:25  0xdb  0x0d
14606    601  1   1789080419=22:46:59  1789080134=22:42:14  0xdb  0x0d  <- the crashing read
16839    633  1   1789080419=22:46:59  1789080134=22:42:14  0xdb  0x09
100      25   3   1788988591           1788986900           0xdb  0x0d

Three of the four share disagg_lsn = 22:46:59, the step-down checkpoint LSN, across two different files — consistent with the force-clear firing only on outdated disaggregated btrees, i.e. only at step-down.

Why the read path cannot catch it

WT_BLOCK_DISAGG_MODIFIED suppresses the cookie checksum comparison, the base_lsn/delta-count consistency asserts and the cumulative_size assert (block_disagg_read.c:219-271, 314), and delta_count is taken from get_args rather than from the number of blocks actually returned. See the existing TODO(WT-16511) at block_disagg_read.c:216.

Suggested fix

Minimal: in _evict_page_victim_cache_eligible(), reject when !wt_page_evict_clean(page) rather than when _wt_page_is_modified(page). This matches the check's stated intent and excludes any page whose dsk no longer corresponds to its block_meta.

Better cache retention: when mod->rec_result == WT_PM_REC_REPLACE, cache the reconciliation image instead of page->dsk — this needs confirmation that the new image is actually retained in memory at that point, since for REPLACE mod_replace holds an address cookie rather than the image.