-
Type:
Build Failure
-
Resolution: Fixed
-
Priority:
Critical - P2
-
Affects Version/s: None
-
Component/s: Test Format
-
Storage Engines, Storage Engines - Transactions
-
SE Transactions - 2025-09-26, SE Transactions - 2025-10-10
-
8
-
6
-
v8.2, v8.0, v7.0
format-stress-test-2 on ubuntu2004-arm64-release-nonstandalone
Host: i-08f5eacfbd5726817
Project: wiredtiger
Commit: ed5d7743
Please refer to BF(G) Playbook for instructions on handling BF and BFG tickets as well as Auto-Resolution Rules
Task Logs:
Logs:
mirror: 1379161/1379160 mismatch: table:T00002: {0001379161.00/opqrstuv}/{0001379161/CMNOPQRSTUV} [0x43], table:T00001: {#}/{0x69} (checkpoint mine.3)
Logs:
t: FAILED: table_verify_mirror/397: base_keyno == table_keyno || (FLD_ISSET(g.trace_flags, TRACE_MIRROR_FAIL) && failures < 20)
Logs:
t: process aborting WiredTiger Error: __wt_abort, 29: aborting WiredTiger library
Logs:
test/format run configuration highlights
Logs:
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x0000ffff89953aac in __GI_abort () at abort.c:79 #2 0x0000ffff89b4d948 in __wt_abort (session=session@entry=0x0) at /data/mci/c37ef1998d0accfb89f5c9d8219847d1/wiredtiger/src/os_common/os_abort.c:31 #3 0x00000000004210bc in testutil_die (e=e@entry=0, fmt=fmt@entry=0x423480 "%s/%d: %s") at /data/mci/c37ef1998d0accfb89f5c9d8219847d1/wiredtiger/test/utility/misc.c:75 #4 0x000000000041989c in table_verify_mirror (conn=conn@entry=0x1052bfe4af00, base=0x1052bfe34000, table=0x1052bfe32000, checkpoint=checkpoint@entry=0xffff7fdae3e5 "mine.3", tinfo=tinfo@entry=0x0) at /data/mci/c37ef1998d0accfb89f5c9d8219847d1/wiredtiger/test/format/verify.c:397 #5 0x000000000041a1e8 in wts_verify_mirrors (conn=<optimized out>, checkpoint=<optimized out>, tinfo=<optimized out>) at /data/mci/c37ef1998d0accfb89f5c9d8219847d1/wiredtiger/test/format/verify.c:504 #6 wts_verify_mirrors (conn=conn@entry=0x1052bfe4af00, checkpoint=checkpoint@entry=0xffff7fdae3e5 "mine.3", tinfo=tinfo@entry=0x0) at /data/mci/c37ef1998d0accfb89f5c9d8219847d1/wiredtiger/test/format/verify.c:492 #7 0x0000000000405b10 in checkpoint (arg=<optimized out>) at /data/mci/c37ef1998d0accfb89f5c9d8219847d1/wiredtiger/test/format/checkpoint.c:164 #8 0x0000ffff89ad9624 in start_thread (arg=0x4059e0 <checkpoint>) at pthread_create.c:477 #9 0x0000ffff89a0466c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78
Repro Artifacts:
Root cause:
When a page containing only tombstone records is written to disk during precise checkpointing or eviction while in a prepared state, the page's time aggregate stores the prepared stop timestamp (stop_prepare_ts) as both newest_stop_ts and newest_stop_durable_ts.
During cursor traversal, the page skipping logic can skip checking a page if the time aggregate's newest_stop_durable_ts is less than checkpoint/stable timestamp. This causes the algorithm to skip pages containing prepared tombstones, even though these prepared tombstones should remain visible at this point until the rollback/commit ts is stable.
The fix is to add a check for the time aggregate's prepare state and never skip if we know a page contains a prepared update.
- is duplicated by
-
WT-15407 test/format key mismatch (non-disagg) rows store
-
- Closed
-
- is related to
-
WT-15238 Found an update with max stop timestamp when inserting data to the history store
-
- Closed
-
-
WT-15575 Prevent the follower from writing checkpoint metadata
-
- Closed
-
-
WT-15563 Investigate making cache tolerant to change app step-wise eviction to incremental eviction
-
- Closed
-
-
WT-15568 Add the ability to dump the error log without the connection object
-
- Closed
-
-
WT-15576 Use EIO to indicate I/O errors in disagg
-
- Closed
-
-
WT-15410 test/format (disagg.mode=leader) Delta chain validation failed
-
- Closed
-
-
WT-15408 Fix prepare insert rollback with an onpage value that has a globally visible stop time point
-
- Closed
-
-
WT-15550 layered38, 32 data mismatch error
-
- Closed
-
-
WT-15566 Remove table_id parameter from palm_handle_get_page_ids
-
- Closed
-
- related to
-
WT-15292 disagg: test_bug033.py, test_prepare{22,23,27}.py crash in __wt_tree_modify_set called by rollback_to_stable
-
- Closed
-
-
WT-15193 Add test case to ensure prefix suffix compression works for page deltas
-
- Closed
-
-
WT-15563 Investigate making cache tolerant to change app step-wise eviction to incremental eviction
-
- Closed
-
-
WT-15407 test/format key mismatch (non-disagg) rows store
-
- Closed
-
-
WT-15410 test/format (disagg.mode=leader) Delta chain validation failed
-
- Closed
-
-
WT-15450 test/format (disagg.mode=leader) verify failure unpacking addr
-
- Closed
-
-
WT-15550 layered38, 32 data mismatch error
-
- Closed
-
-
WT-15566 Remove table_id parameter from palm_handle_get_page_ids
-
- Closed
-