-
Type:
Bug
-
Resolution: Fixed
-
Priority:
Minor - P4
-
Affects Version/s: None
-
Component/s: Block Manager
-
None
-
Storage Engines, Storage Engines - Persistence
-
0.017
-
StorEng - Defined Pipeline
-
None
__wti_block_extlist_merge() merges one block-manager extent list into another. When the source list is larger than the destination it swaps the two lists first, so the merge walks the shorter one. The swap exchanges the skiplist heads and the bookkeeping counters, but not the cached end-of-list pointer. The destination is left pointing at an extent that now belongs to the source list, which the caller frees.
How it works today
WT_EXTLIST.last caches the last extent on the by-offset skiplist. It is set when an extent is appended at the end of a list, cleared when the last extent is removed, and read by the append path so that growing the list is O(1) rather than a skiplist walk. It is a live pointer into the list, not just a hint: the append path dereferences it and can grow the extent it points at.
__wti_block_extlist_merge() swaps the two lists when the source has more entries than the destination and both lists track sizes the same way. The swap moves the skiplist heads, and the bytes, entries and (as of WT-16796) size_buckets counters, so that each list's bookkeeping ends up describing the skiplist it now owns. The cached end pointer is not part of the swap.
The gap
After a swap the destination's cached end pointer still points at the destination's original last extent, which now sits on the source's skiplist; the source's pointer points into the destination's list the same way. Neither list's cached pointer describes the list it is attached to.
This matters because the lists that reach the swap without tracking sizes are the allocation and discard lists, and those are the lists that use the append path that reads the cached pointer. Callers free the source list immediately after the merge, so the destination is left holding a pointer into freed memory.
Impact
The append path dereferences the cached pointer once per call. If the pointer is stale, it is a read of freed memory. The append path then recomputes the real end of the list and stores it back, so the cache is repaired by the first append that takes the slow path; the exposure is a single call.
Today that read does not change behaviour, and it is worth being precise about why, because it is not obvious from the merge alone. The stale extent is the destination's original last extent, and the merge folds its range back into the destination, so the destination's list always ends at or beyond that extent. An append at the file end is at or past the end of the destination's list, so the test the append path makes against the stale extent cannot succeed: the code falls through to the correct path and repairs the pointer. The one case where the stale extent does end exactly where the append lands is the case where the merge itself already replaced the pointer, because then the merged range became the new end of the list. Both branches are confirmed by unit tests.
What is left is a use-after-free read: undefined behaviour that aborts under the sanitizer builds, and a violation of the invariant that a list's cached end pointer describes its own skiplist. It is also fragile. The benign outcome depends on an emergent relationship between the two lists rather than on anything stated in the code, so a change to the append path, or a new reader of the cached pointer, turns it into a use-after-free write, and the destination's list silently loses the appended range.
File verification reaches the same swap, but its cached pointer is never dereferenced: the verification list is built by the merge path, not the append path. There the stale pointer is inert.
Reproducer
A Catch2 unit test drives the merge with a source larger than the destination, then checks that the destination's cached end pointer is either unset or reachable from the destination's own skiplist. It fails on current develop.
Proposed fix
Swap the cached end pointer along with the rest of the list in __wti_block_extlist_merge(). Clearing both pointers would also be correct, since the cache is recomputed on demand, but swapping keeps it warm.