-
Type:
Bug
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Disagg CI-blocker, Test Format
-
None
-
Storage Engines, Storage Engines - Transactions
-
SE Transactions - 2025-09-12
-
5
We’ve been seeing an intermittent failure in test/format related to prepare transactions in disaggregated storage. I investigated further with the config in the comments and modified the and code slightly to make it reproduce consistently.
diff --git a/test/format/ops.c b/test/format/ops.c
index 4a0f16aa1a..17610d66cf 100644
--- a/test/format/ops.c
+++ b/test/format/ops.c
@@ -1704,7 +1704,7 @@ read_row(TINFO *tinfo)
{
/* 25% of the time we call search-near. */
return (read_row_worker(tinfo, NULL, tinfo->cursor, tinfo->keyno, tinfo->key, tinfo->value,
- &tinfo->bitv, mmrand(&tinfo->extra_rnd, 0, 3) == 1));
+ &tinfo->bitv, true));
}
Reproduction steps:
- Run format with the config and code changes above
Issue summary:
During a search near, if format hits a prepare conflict in a disagg storage scenario (format_inline.h:57), it retries as expected. However, in disagg case, the key ends up being unset during the retry, causing search near to fail with:
t: process 517857 running
t: WARNING: table selection didn't support mirroring, turning off mirroring
t: WARNING: Setting cache.eviction_dirty_trigger=95 and cache.eviction_update_trigger=95. In disaggregated follower mode, these eviction trigger thresholds are increased to help avoid operation thread stalls.
0xfcc2c57f6ad0:[1753098464:391552][517857:0xfcc2c59c3020], t, txn-recover: [WT_VERB_RTS][WARNING]: skipped recovery RTS due to disagg
0xfcc2c57f5010:[1753098465:392875][517857:0xfcc2c59c3020], t, WT_CONNECTION.close: [WT_VERB_RTS][WARNING]: skipped shutdown RTS due to disagg
0xc2c7bcc88440:[1753098466:387556][517857:0xfcc2c59c3020], t, txn-recover: [WT_VERB_RTS][WARNING]: skipped recovery RTS due to disagg
[1753098466:447387][517857:0xfcc22cc0f120], t, layered:T00001, WT_CURSOR.search_near: [WT_VERB_DEFAULT][ERROR]: __wt_cursor_kv_not_set, 240: requires key be set: Invalid argument
rets = 22
t: FAILED: read_op/111: false
Next steps:
Investigate why the key is unset after the retry and ensure search near handles prepare conflicts correctly in disagg.
- related to
-
WT-14931 Add preserve prepare config tests to test format and test checkpoint
-
- In Progress
-