-
Type:
Task
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: Test Format
-
None
-
Storage Engines - Transactions
-
112.525
-
SE Transactions - 2026-11-06
-
3
test/format runs mirrored tables – row-store and column-store tables holding identical logical data, specifically to catch storage-engine bugs invisible at the data level. Until now, nothing checked that a mirrored operation actually behaved identically across the group at the moment it ran: a remove or blind modify that returns not-found on one mirror but succeeds on another went unnoticed, with the divergence only surfacing later, far from its cause, as an unrelated periodic mirror-verify mismatch.
Each mirror table's result for modify and remove is now recorded and compared against the rest of the group, dying immediately on a disagreement instead of leaving it to surface later. A snapshot-isolation read gets the same check, plus a value-level comparison: two mirrors can agree a key exists yet disagree on its value, which the periodic mirror verify wouldn't catch until much later.
Read comparisons are scoped carefully to avoid false positives: only snapshot-isolation reads are compared, since read-committed and read-uncommitted have no fixed snapshot and each read call can legitimately observe a different, more-recent commit; and a bounded read is excluded entirely, since its search window is derived from each table's own row count and re-randomized per call, so the same key can fall inside one mirror's bound and outside another's without any real divergence.
Verified with a broad Evergreen run across the format-stress and disagg format-stress task groups (23/23 tasks passing).