failed: format-stress-test-disagg-switch-1 on ubuntu2004-stress-tests-arm64 [wiredtiger @ 8bc80573]

XMLWordPrintableJSON

    • Type: Build Failure
    • Resolution: Duplicate
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Reconciliation

      format-stress-test-disagg-switch-1 on ubuntu2004-stress-tests-arm64

      Host: i-0b65d29b6e826c974
      Project: wiredtiger
      Commit: 8bc80573
      Please refer to BF(G) Playbook for instructions on handling BF and BFG tickets as well as Auto-Resolution Rules

      Task Logs:

      format-stress-test-disagg-switch-1 task_log

      Logs:

      [410/669] Building C object test/csuite/CMakeFiles/test_wt16990_disagg_checkpoint_panic.dir/wt16990_disagg_checkpoint_panic/main.c.o
[418/669] Linking CXX executable test/csuite/wt16990_disagg_checkpoint_panic/test_wt16990_disagg_checkpoint_panic

      logs

      format-stress-test-disagg-switch-1 task_log

      Logs:

      [1779845575:333293][24834:0xffffa9b96c80], t, file:T00002.wt_stable, eviction-server: [WT_VERB_EXTENSION][ERROR]: ext/page_log/palite/palite.cpp:1877: Pages::verify_chain(const std::vector<PageInfo>&, uint64_t)::<lambda(const PageInfo&)>: Multiple discarded pages in chain: {table_id=41, page_id=410, lsn=269641, backlink_lsn=3319, base_lsn=3319, flags=0x10002}, {table_id=41, page_id=410, lsn=286342, backlink_lsn=3319, base_lsn=3319, flags=0x10002}
[1779845575:333409][24834:0xffffa9b96c80], t, file:T00002.wt_stable, eviction-server: [WT_VERB_EXTENSION][ERROR]: ext/page_log/palite/palite.cpp:575: int safe_call(WT_SESSION*, S*, MemberFunc, Args&& ...) [with T = PaliteHandle; S = __wt_page_log_handle; MemberFunc = int (PaliteHandle::*)(long unsigned int, long unsigned int, __wt_page_log_discard_args*); Args = {long unsigned int&, long unsigned int&, __wt_page_log_discard_args*&}; WT_SESSION = __wt_session]: Call failed
[1779845575:333420][24834:0xffffa9b96c80], t, file:T00002.wt_stable, eviction-server: [WT_VERB_ERROR_RETURNS][ERROR]: __wt_try_readlock, 151: Error at src/support/mtx_rw.c:151: "__wt_atomic_cas_uint64_v(&l->u.v, old.u.v, new.u.v) ? 0 : EBUSY" failed: Device or resource busy
[1779845575:333430][24834:0xffffa9b96c80], t, file:T00002.wt_stable, eviction-server: [WT_VERB_ERROR_RETURNS][ERROR]: __wt_try_readlock, 151: Error at src/support/mtx_rw.c:151: "__wt_atomic_cas_uint64_v(&l->u.v, old.u.v, new.u.v) ? 0 : EBUSY" failed: Device or resource busy
[1779845575:333433][24834:0xffffa9b96c80], t, file:T00002.wt_stable, eviction-server: [WT_VERB_ERROR_RETURNS][ERROR]: __wt_try_readlock, 151: Error at src/support/mtx_rw.c:151: "__wt_atomic_cas_uint64_v(&l->u.v, old.u.v, new.u.v) ? 0 : EBUSY" failed: Device or resource busy
[1779845575:333436][24834:0xffffa9b96c80], t, file:T00002.wt_stable, eviction-server: [WT_VERB_ERROR_RETURNS][ERROR]: __wt_ref_block_free, 1915: Error at src/include/btree_inline.h:1915: "__wt_btree_block_free(session, addr.addr, addr.size)" failed: Invalid argument
[1779845575:333441][24834:0xffffa9b96c80], t, file:T00002.wt_stable, eviction-server: [WT_VERB_ERROR_RETURNS][ERROR]: __rec_write_wrapup, 2949: Error at src/reconcile/rec_write.c:2949: "__wt_ref_block_free(session, ref, disagg_page_free_required)" failed: Invalid argument
[1779845575:333444][24834:0xffffa9b96c80], t, file:T00002.wt_stable, eviction-server: [WT_VERB_ERROR_RETURNS][ERROR]: __reconcile, 398: Error at src/reconcile/rec_write.c:398: "__rec_write_wrapup(session, r)" failed: Invalid argument
[1779845575:333447][24834:0xffffa9b96c80], t, file:T00002.wt_stable, eviction-server: [WT_VERB_DEFAULT][ERROR]: __reconcile, 452: reconciliation failed after building the disk image: Invalid argument
[1779845575:333449][24834:0xffffa9b96c80], t, file:T00002.wt_stable, eviction-server: [WT_VERB_DEFAULT][ERROR]: __reconcile, 452: the process must exit and restart: WT_PANIC: WiredTiger library panic
[1779845575:333453][24834:0xffffa9b96c80], t, file:T00002.wt_stable, eviction-server: [WT_VERB_DEFAULT][ERROR]: __wt_abort, 29: aborting WiredTiger library

      logs

      format-stress-test-disagg-switch-1 task_log

      Logs:

      #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x0000ffffaba6baac in __GI_abort () at abort.c:79
#2  0x0000ffffabf47950 in __wt_abort (session=session@entry=0x12b2e92e4000) at /data/mci/9eddd5de07448e32c07d790051232ab0/wiredtiger/src/os_common/os_abort.c:32
#3  0x0000ffffabff1e0c in __wt_panic_func (session=session@entry=0x12b2e92e4000, error=error@entry=22, func=func@entry=0xffffac0fa1b0 <__PRETTY_FUNCTION__.66> "__reconcile", line=line@entry=452, category=category@entry=WT_VERB_DEFAULT, fmt=fmt@entry=0xffffac0ae150 "reconciliation failed after building the disk image") at /data/mci/9eddd5de07448e32c07d790051232ab0/wiredtiger/src/support/err.c:633
#4  0x0000ffffabf97158 in __reconcile (session=session@entry=0x12b2e92e4000, ref=ref@entry=0x12b32d2fd900, salvage=salvage@entry=0x0, flags=flags@entry=672, page_lockedp=page_lockedp@entry=0xffffa9b9519f) at /data/mci/9eddd5de07448e32c07d790051232ab0/wiredtiger/src/reconcile/rec_write.c:452
#5  0x0000ffffabf95b18 in __wt_reconcile (session=session@entry=0x12b2e92e4000, ref=ref@entry=0x12b32d2fd900, salvage=salvage@entry=0x0, flags=flags@entry=672) at /data/mci/9eddd5de07448e32c07d790051232ab0/wiredtiger/src/reconcile/rec_write.c:127
#6  0x0000ffffabef8e84 in __evict_reconcile (session=session@entry=0x12b2e92e4000, ref=ref@entry=0x12b32d2fd900, evict_flags=evict_flags@entry=0) at /data/mci/9eddd5de07448e32c07d790051232ab0/wiredtiger/src/evict/evict_page.c:1277
#7  0x0000ffffabefb5c8 in __wt_evict (session=session@entry=0x12b2e92e4000, ref=0x12b32d2fd900, previous_state=3 '\003', flags=flags@entry=0) at /data/mci/9eddd5de07448e32c07d790051232ab0/wiredtiger/src/evict/evict_page.c:443
#8  0x0000ffffabef03a8 in __wti_evict_page (session=session@entry=0x12b2e92e4000, is_server=is_server@entry=false) at /data/mci/9eddd5de07448e32c07d790051232ab0/wiredtiger/src/evict/evict_dispatch.c:254
#9  0x0000ffffabefc2a8 in __wti_evict_lru_pages (session=session@entry=0x12b2e92e4000, is_server=is_server@entry=false) at /data/mci/9eddd5de07448e32c07d790051232ab0/wiredtiger/src/evict/evict_queue.c:140
#10 0x0000ffffabf001d0 in __evict_thread_run (session=0x12b2e92e4000, thread=0x12b2fd912c80) at /data/mci/9eddd5de07448e32c07d790051232ab0/wiredtiger/src/evict/evict_thread.c:117
#11 0x0000ffffac00da70 in __thread_run (arg=0x12b2fd912c80) at /data/mci/9eddd5de07448e32c07d790051232ab0/wiredtiger/src/support/thread_group.c:32
#12 0x0000ffffabbc5624 in start_thread (arg=0xffffac00da1c <__thread_run>) at pthread_create.c:477
#13 0x0000ffffabb1c66c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78

      logs

      Repro Artifacts:

            Assignee:
            Shoufu Du
            Reporter:
            xgen-buildbaron-user
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: