Motivation

Layered cursors are one of the most complex components in the engine, yet today their only automated coverage is the Python integration suite. A component this intricate deserves diverse, deep testing across several levels, and unit tests are the layer we are currently missing. This ticket adds a grey-box unit-test suite (built on Catch2) that drives the layered cursor directly below the public API, so its behaviour can be validated in isolation rather than only end-to-end. The aim is not to duplicate the integration tests but to round out the testing pyramid for a component whose correctness is hard to reason about from the outside.

The hard part is that a layered cursor's behaviour depends on many state dimensions at once – the table content (what lives in ingest vs. stable, value overrides, tombstones), the cursor's position and iteration direction, its internal/inter-API state, transaction and visibility state, and more. That combinatorial space is far too large to cover by hand.

The first design idea is to solve the problem around the tables content and positioning dimensions by generating a large population of tables up front, enumerating the full range of content and position combinations, so coverage of those axes comes from generation rather than hand-written cases. The second idea keeps this affordable: we deliberately scope commit-time visibility out and run every operation inside a transaction that is rolled back, never committed. That keeps the generated tables pristine and reusable across cases and removes the cost of building and tearing down committed state on every case; commit and durability semantics remain the integration suite's responsibility.

Another challenge is how we manager testing inter API relations (when we call search() first, then next()). The idea is that the cursor's internal state on entry to each API call – is handled in two steps: first the cursor is driven into a chosen incoming state through real API calls (a distinct setup step), then the operation under test runs against it. The verification side is what makes this worthwhile: rather than checking only the returned key, value, and error code, each case asserts the full internal state – the constituent ingest and stable cursors, their positions and content, and the layered cursor's flags. The guiding principle is that a layered cursor must behave exactly like the equivalent plain (non-layered) cursor over the same merged content, so any divergence in that internal state is a bug to surface rather than behaviour to accommodate.

This grey-box validation is precisely what lets the unit tests catch issues a black-box test cannot see and provide verification based on internal contracts between different cursor APIs (what method should leave a cursor in what internal state (ingest, stable, current, iface cursors positioned, flags, etc) instead of testing all the multiplications of what API could be called after a different API.

All the states covered by this testing (or intentionally left out of scope) should be documented in some form, such as a design document or a state diagram.

The main challenges discovered so far:

• A significant amount of AI-generated code that requires careful review.
• Existing issues being uncovered during the process, which need to be investigated and addressed along the way.