-
Type:
Build Failure
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Storage Engines, Storage Engines - Persistence
-
None
-
None
-
1
format-stress-test-disagg-leader-pull-request on ubuntu2004-ubsan
Host: i-0762f06897f6e5ed0
Project: wiredtiger
Commit: 27ea5d1a
Please refer to BF(G) Playbook for instructions on handling BF and BFG tickets as well as Auto-Resolution Rules
Task Logs:
format-stress-test-disagg-leader-pull-request task_log
Logs:
[1771301831:056643][13120:0x7f0b2810b700], t, layered:T00002, WT_SESSION.verify: [WT_VERB_DEFAULT][ERROR]: int __schema_layered_worker_verify(WT_SESSION_IMPL *, const char *, int (*)(WT_SESSION_IMPL *, const char **), int (*)(WT_SESSION_IMPL *, const char *, _Bool *), const char **, uint32_t), 132: WiredTiger assertion failed: 'ingest_ret != 16'. Verify: file:T00002.wt_ingest ingest table on leader cannot be verified. Ingest contains dirty content or open cursors, which is an invalid state.
[1771301831:056684][13120:0x7f0b2810b700], t, layered:T00002, WT_SESSION.verify: [WT_VERB_DEFAULT][ERROR]: void __wt_abort(WT_SESSION_IMPL *), 29: aborting WiredTiger library
[1771301831:056695][13120:0x7f0b2810b700], t, layered:T00002, WT_SESSION.verify: [WT_VERB_ERROR_RETURNS][ERROR]: int __wt_btcur_prev(WT_CURSOR_BTREE *, _Bool), 706: Error at src/btree/bt_curprev.c:706: "WT_NOTFOUND" failed: WT_NOTFOUND: item not found
[1771301831:056707][13120:0x7f0b2810b700], t, layered:T00002, WT_SESSION.verify: [WT_VERB_ERROR_RETURNS][ERROR]: int __curfile_prev(WT_CURSOR *), 246: Error at src/cursor/cur_file.c:246: "ret" failed: WT_NOTFOUND: item not found
[1771301831:056713][13120:0x7f0b2810b700], t, layered:T00002, WT_SESSION.verify: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_search_near(WT_CURSOR *, int *), 879: Error at src/cursor/cur_hs.c:879: "ret" failed: WT_NOTFOUND: item not found
[1771301831:056719][13120:0x7f0b2810b700], t, layered:T00002, WT_SESSION.verify: [WT_VERB_ERROR_RETURNS][ERROR]: int __curhs_search_near_helper(WT_SESSION_IMPL *, WT_CURSOR *, _Bool), 751: Error at src/cursor/cur_hs.c:751: "cursor->search_near(cursor, &cmp)" failed: WT_NOTFOUND: item not found
[1771301831:056725][13120:0x7f0b2810b700], t, layered:T00002, WT_SESSION.verify: [WT_VERB_ERROR_RETURNS][ERROR]: int __wt_session_get_dhandle(WT_SESSION_IMPL *, const char *, const char *, const char **, uint32_t), 939: Error at src/session/session_dhandle.c:939: "__wt_session_lock_dhandle(session, flags, &is_dead)" failed: Device or resource busy
[1771301831:056736][13120:0x7f0b2810b700], t, layered:T00002, WT_SESSION.verify: [WT_VERB_ERROR_RETURNS][ERROR]: int __conn_dhandle_close_one(WT_SESSION_IMPL *, const char *, const char *, _Bool, _Bool, _Bool), 846: Error at src/conn/conn_dhandle.c:846: "__wt_session_get_dhandle( session, uri, checkpoint, NULL, WT_DHANDLE_EXCLUSIVE | WT_DHANDLE_LOCK_ONLY)" failed: Device or resource busy
[1771301831:056743][13120:0x7f0b2810b700], t, layered:T00002, WT_SESSION.verify: [WT_VERB_ERROR_RETURNS][ERROR]: int __wt_conn_dhandle_close_all(WT_SESSION_IMPL *, const char *, _Bool, _Bool, _Bool), 896: Error at src/conn/conn_dhandle.c:896: "__conn_dhandle_close_one(session, uri, NULL, removed, mark_dead, check_visibility)" failed: Device or resource busy
[1771301831:056749][13120:0x7f0b2810b700], t, layered:T00002, WT_SESSION.verify: [WT_VERB_ERROR_RETURNS][ERROR]: int __wti_execute_handle_operation(WT_SESSION_IMPL *, const char *, int (*)(WT_SESSION_IMPL *, const char **), const char **, uint32_t), 28: Error at src/schema/schema_worker.c:28: "ret" failed: Device or resource busy
[1771301831:056755][13120:0x7f0b2810b700], t, layered:T00002, WT_SESSION.verify: [WT_VERB_ERROR_RETURNS][ERROR]: int __wt_schema_worker(WT_SESSION_IMPL *, const char *, int (*)(WT_SESSION_IMPL *, const char **), int (*)(WT_SESSION_IMPL *, const char *, _Bool *), const char **, uint32_t), 191: Error at src/schema/schema_worker.c:191: "__wti_execute_handle_operation(session, uri, file_func, cfg, open_flags)" failed: Device or resource busy
format-stress-test-disagg-leader-pull-request task_log
Logs:
test/format run configuration highlights
format-stress-test-disagg-leader-pull-request task_log
Logs:
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x00007f0b28132859 in __GI_abort () at abort.c:79 #2 0x00007f0b29dfe615 in __wt_abort (session=0x55c9bd23f6f0) at /data/mci/22b839681c011b3146f147be6bebd2ff/wiredtiger/src/os_common/os_abort.c:32 #3 0x00007f0b2a0fc155 in __schema_layered_worker_verify (session=0x55c9bd23f6f0, uri=0x55c9bdce1fd0 "layered:T00002", file_func=0x7f0b295bc908 <__wt_verify>, name_func=0x0, cfg=0x7ffc41d83220, open_flags=16777248) at /data/mci/22b839681c011b3146f147be6bebd2ff/wiredtiger/src/schema/schema_worker.c:128 #4 0x00007f0b2a0fab31 in __wt_schema_worker (session=0x55c9bd23f6f0, uri=0x55c9bdce1fd0 "layered:T00002", file_func=0x7f0b295bc908 <__wt_verify>, name_func=0x0, cfg=0x7ffc41d83220, open_flags=16777248) at /data/mci/22b839681c011b3146f147be6bebd2ff/wiredtiger/src/schema/schema_worker.c:247 #5 0x00007f0b2a0fa587 in __wt_schema_worker (session=0x55c9bd23f6f0, uri=0x55c9bcfd9414 "table:T00002", file_func=0x7f0b295bc908 <__wt_verify>, name_func=0x0, cfg=0x7ffc41d83220, open_flags=16777248) at /data/mci/22b839681c011b3146f147be6bebd2ff/wiredtiger/src/schema/schema_worker.c:225 #6 0x00007f0b2a14a6da in __session_verify (wt_session=0x55c9bd23f6f0, uri=0x55c9bcfd9414 "table:T00002", config=0x55c9ba87c76b "strict") at /data/mci/22b839681c011b3146f147be6bebd2ff/wiredtiger/src/session/session_api.c:1817 #7 0x000055c9ba83034d in table_verify () #8 0x000055c9ba8311fd in tables_apply () #9 0x000055c9ba830965 in wts_verify () #10 0x000055c9ba82ce33 in main ()
Repro Artifacts: