-
Type:
Build Failure
-
Resolution: Gone away
-
Priority:
Critical - P2
-
None
-
Affects Version/s: None
-
Component/s: Btree
-
Storage Engines - Transactions
-
257.041
-
SE Transactions - 2026-07-31
-
5
-
1
unit-test-hook-disagg-leader-table-bucket00 on amazon2023-arm64-asan
Host: i-0533f1a92fcd30225
Project: wiredtiger
Commit: f4c65e37
Please refer to BF(G) Playbook for instructions on handling BF and BFG tickets as well as Auto-Resolution Rules
Task Logs:
Logs:
/opt/mongodbtoolchain/revisions/93d85cc1a00ca1d53fcc7d4ca790f19a4e5dd542/scripts/products.sh: line 13: /opt/mongodbtoolchain/revisions/93d85cc1a00ca1d53fcc7d4ca790f19a4e5dd542/scripts/installer.sh: No such file or directory
Logs:
[pid:12365]: Publishing CREATE and DROP at different epochs is an API violation that panics. (palite) ... skipped 'test_layered_schema10.test_layered_schema10.test_create_drop_split_epochs -s 0 (palite) for hook disagg: Layered tests already turn on the proper stuff'
Logs:
#0 0x0000ffff95ac4454 in __pthread_kill_implementation () from /lib64/libc.so.6 #0 0x0000ffff95ac4454 in __pthread_kill_implementation () from /lib64/libc.so.6 #1 0x0000ffff95a7b320 [PAC] in raise () from /lib64/libc.so.6 #2 0x0000ffff95a62224 [PAC] in abort () from /lib64/libc.so.6 #3 0x0000ffff912bb794 [PAC] in __wt_abort (session=0xffff88e380a0) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/os_common/os_abort.c:32 #4 0x0000ffff9159f754 in __wt_panic_func (session=0xffff88e380a0, error=22, func=0xffff917a3260 <__PRETTY_FUNCTION__.__cell_check_value_validity> "int __cell_check_value_validity(WT_SESSION_IMPL *, WT_TIME_WINDOW *, _Bool)", line=23, category=WT_VERB_DEFAULT, fmt=0xffff917a32e0 <str> "value timestamp window failed validation") at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/support/err.c:633 #5 0x0000ffff90c0b940 in __cell_check_value_validity (session=0xffff88e380a0, tw=0x50c000b2b728, expected_error=false) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/include/cell_inline.h:23 #6 0x0000ffff90c09a48 in __cell_pack_value_validity (session=0xffff88e380a0, pp=0xffff8564e820, tw=0x50c000b2b728) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/include/cell_inline.h:67 #7 0x0000ffff90c084f8 in __wt_cell_pack_value (session=0xffff88e380a0, cell=0xffff8564d928, tw=0x50c000b2b728, rle=0, size=504) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/include/cell_inline.h:647 #8 0x0000ffff90bf3ea4 in __wt_cell_pack_leaf_kv (session=0xffff88e380a0, empty_value=false, key_data=0x52b0069c91b2, key_size=6, val_data=0x52b0069c91c5, val_size=504, val_tw=0x50c000b2b728, new_image=0xffff859791e0, s=0xffff8588d0a0) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/include/cell_inline.h:386 #9 0x0000ffff90bec280 in __wti_page_merge_deltas_with_base_image_leaf (session=0xffff88e380a0, deltas=0x507002743ef8, delta_size=1, new_image=0xffff859791e0, base_dsk=0x52b0069c4200, ta=0xffff8597a140) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/btree/bt_page.c:377 #10 0x0000ffff90c55298 in __page_read_build_full_disk_image (session=0xffff88e380a0, deltas=0x507002743ef8, delta_size=1, new_image=0xffff859791e0, base_image_addr=0x52b0069c4200, ta=0xffff8597a140) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/btree/bt_read.c:173 #11 0x0000ffff90c4d3ac in __page_read (session=0xffff88e380a0, ref=0x50e002169320, flags=256) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/btree/bt_read.c:365 #12 0x0000ffff90c492e8 in __wt_page_in_func (session=0xffff88e380a0, ref=0x50e002169320, flags=256, func=0xffff917e8140 <__PRETTY_FUNCTION__.__wt_row_search> "int __wt_row_search(WT_CURSOR_BTREE *, WT_ITEM *, _Bool, WT_REF *, _Bool, _Bool *)", line=576) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/btree/bt_read.c:610 #13 0x0000ffff90de7644 in __wt_page_swap_func (session=0xffff88e380a0, held=0x51800008c9b0, want=0x50e002169320, flags=256, func=0xffff917e8140 <__PRETTY_FUNCTION__.__wt_row_search> "int __wt_row_search(WT_CURSOR_BTREE *, WT_ITEM *, _Bool, WT_REF *, _Bool, _Bool *)", line=576) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/include/btree_inline.h:2680 #14 0x0000ffff90de2c48 in __wt_row_search (cbt=0x51a000427880, srch_key=0x51a0004279a0, insert=true, leaf=0x0, leaf_safe=false, leaf_foundp=0x0) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/btree/row_srch.c:576 #15 0x0000ffff90b30694 in __cursor_row_search (cbt=0x51a000427880, insert=true, leaf=0x0, leaf_foundp=0x0) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/btree/bt_cursor.c:512 #16 0x0000ffff90b2a73c in __cursor_search (cbt=0x51a000427880, leaf=0x0, leaf_foundp=0x0, insert=true) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/btree/bt_cursor.c:528 #17 0x0000ffff90b2b8cc in __wt_btcur_search_near (cbt=0x51a000427880, exactp=0xffff858850c0) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/btree/bt_cursor.c:975 #18 0x0000ffff90fde770 in __curfile_search_near (cursor=0x51a000427880, exact=0xffff858850c0) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/cursor/cur_file.c:366 #19 0x0000ffff9102d91c in __curhs_file_cursor_search_near (session=0xffff88e380a0, cursor=0x51a000427880, exactp=0xffff858850c0) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/cursor/cur_hs.c:226 #20 0x0000ffff9100b1c4 in __curhs_search_near (cursor=0x51600d43d680, exactp=0xffff851e16c0) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/cursor/cur_hs.c:816 #21 0x0000ffff90ffede4 in __curhs_search_near_helper (session=0xffff88e380a0, cursor=0x51600d43d680, before=true) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/cursor/cur_hs.c:745 #22 0x0000ffff90ffeac8 in __wt_curhs_search_near_before (session=0xffff88e380a0, cursor=0x51600d43d680) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/cursor/cur_hs.c:719 #23 0x0000ffff9145be54 in __rec_hs_insert_record (session=0xffff88e380a0, cursor=0x51600d43d680, btree=0x5180004d2480, ref=0x50e0023655a0, key=0x5040071c91d0, type=3 '\003', hs_value=0x5040046ad4d0, tw=0xffff85883060, error_on_ts_ordering=false) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/reconcile/rec_hs.c:439 #24 0x0000ffff914590c0 in __rec_hs_write_upd (session=0xffff88e380a0, hs_cursor=0x51600d43d680, btree=0x5180004d2480, ref=0x50e0023655a0, key=0x5040071c91d0, upd=0x51600d5d8080, newest_hs=0x51600d5d8080, full_value=0x5040046ad4d0, prev_full_value=0x5040071b6dd0, tw=0xffff85883060, enable_reverse_modify=true, error_on_ts_ordering=false, modify_cntp=0xffff85883130, statsp=0xffff8567f8a0) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/reconcile/rec_hs.c:1011 #25 0x0000ffff9144b2c0 in __rec_hs_flush_upd_chain (session=0xffff88e380a0, hs_cursor=0x51600d43d680, btree=0x5180004d2480, ref=0x50e0023655a0, key=0x5040071c91d0, updates=0xffff8567f920, list=0x51f000400a80, error_on_ts_ordering=false, key_state=0xffff8567f860, statsp=0xffff8567f8a0) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/reconcile/rec_hs.c:1123 #26 0x0000ffff91445fb8 in __wti_rec_hs_insert_updates (session=0xffff88e380a0, r=0x51e0011e5c80, multi=0x51a0002c8280) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/reconcile/rec_hs.c:1285 #27 0x0000ffff91436594 in __rec_hs_wrapup (session=0xffff88e380a0, r=0x51e0011e5c80) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/reconcile/rec_write.c:3461 #28 0x0000ffff914070f8 in __rec_write_wrapup (session=0xffff88e380a0, r=0x51e0011e5c80) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/reconcile/rec_write.c:3006 #29 0x0000ffff913f35dc in __reconcile (session=0xffff88e380a0, ref=0x50e0023655a0, salvage=0x0, flags=672, page_lockedp=0xffff85819020) at /data/mci/c15d824bbe0b457235bb523fbdc9fcd3/wiredtiger/src/reconcile/rec_write.c:391
Logs:
[1784012525:584423][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_DEFAULT][ERROR]: int __wt_time_value_validate(WT_SESSION_IMPL *, WT_TIME_WINDOW *, WT_TIME_AGGREGATE *, _Bool), 487: value time window has a start time after its stop time; time window start: durable_timestamp=(0, 3) timestamp=(0, 3) prepare_timestamp=(0, 0) prepared_id=0 transaction=24648 | stop: durable_timestamp=(0, 0) timestamp=(0, 0) prepare_timestamp=(0, 0) prepared_id=0 transaction=0: Invalid argument [1784012525:584654][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:584708][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:584741][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:584771][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:584802][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:584832][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:584861][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:584889][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:584924][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:584966][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:584997][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:585031][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:585060][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:585090][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:585126][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:585157][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:585185][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:585216][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:585248][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:585281][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:585311][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:585339][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:585369][12365:0xffff81d660c0], test_hs11.test_hs11.test_ts_updates_donot_clears_hs(integer-row.deletion.no-long-running.modify.large-nrows.insert-list), file:WiredTigerSharedHS.wt_stable, WT_CURSOR.search_near: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_next(WT_CURSOR *), 321: Error at src/cursor/cur_hs.c:321: "__curhs_next_visible(session, hs_cursor)" failed: WT_NOTFOUND: item not found [1784012525:585397][12365:0xffff81d660c0], test_hs11.test_hs11.t
- causes
-
WT-17577 Fix disaggregated leaf delta omitting base-image cells from its parent's time aggregate
-
- Closed
-
- fixes
-
WT-17885 Optimize __time_window_clear_obsolete to check stop time point first
-
- Closed
-
-
WT-18162 Skip materializing globally-visible-deleted keys when reconstructing a disaggregated page from base image + deltas
-
- Closed
-
- is related to
-
WT-16523 Verify fails because of timestamps mismatch when writing deltas vs reading deltas
-
- Closed
-
- related to
-
WT-18162 Skip materializing globally-visible-deleted keys when reconstructing a disaggregated page from base image + deltas
-
- Closed
-