failed: format-stress-test-disagg-leader-1 on amazon2023-stress-tests-arm64 [wiredtiger @ 193fb5a2]

XMLWordPrintableJSON

    • Type: Build Failure
    • Resolution: Duplicate
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: PALite

      format-stress-test-disagg-leader-1 on amazon2023-stress-tests-arm64

      Host: i-021329b2fe16c82d2
      Project: wiredtiger
      Commit: 193fb5a2
      Please refer to BF(G) Playbook for instructions on handling BF and BFG tickets as well as Auto-Resolution Rules

      Task Logs:

      format-stress-test-disagg-leader-1 task_log

      Logs:

          [1773903939:624184][16675:0xffff6f9cdbc0], t, file:T00002.wt_stable, WT_SESSION.rollback_transaction: [WT_VERB_EXTENSION][ERROR]: ext/page_log/palite/palite.cpp:1869: Pages::verify_chain(const std::vector<PageInfo>&, uint64_t)::<lambda(const PageInfo&)>: Multiple discarded pages in chain: {table_id=41, page_id=27017, lsn=1554326, backlink_lsn=1554324, base_lsn=789218, flags=0x10002}, {table_id=41, page_id=27017, lsn=1582755, backlink_lsn=1483344, base_lsn=789218, flags=0x10002}
          [1773903939:624276][16675:0xffff6f9cdbc0], t, file:T00002.wt_stable, WT_SESSION.rollback_transaction: [WT_VERB_EXTENSION][ERROR]: ext/page_log/palite/palite.cpp:567: int safe_call(WT_SESSION*, S*, MemberFunc, Args&& ...) [with T = PaliteHandle; S = __wt_page_log_handle; MemberFunc = int (PaliteHandle::*)(long unsigned int, long unsigned int, __wt_page_log_discard_args*); Args = {long unsigned int&, long unsigned int&, __wt_page_log_discard_args*&}; WT_SESSION = __wt_session]: Call failed
          [1773903939:624285][16675:0xffff6f9cdbc0], t, file:T00002.wt_stable, WT_SESSION.rollback_transaction: [WT_VERB_ERROR_RETURNS][ERROR]: __wt_ref_block_free, 1940: Error at src/include/btree_inline.h:1940: "__wt_btree_block_free(session, addr.addr, addr.size)" failed: Invalid argument
          [1773903939:624295][16675:0xffff6f9cdbc0], t, file:T00002.wt_stable, WT_SESSION.rollback_transaction: [WT_VERB_ERROR_RETURNS][ERROR]: __rec_write_wrapup, 2968: Error at src/reconcile/rec_write.c:2968: "__wt_ref_block_free(session, ref, true)" failed: Invalid argument
          [1773903939:624298][16675:0xffff6f9cdbc0], t, file:T00002.wt_stable, WT_SESSION.rollback_transaction: [WT_VERB_ERROR_RETURNS][ERROR]: __reconcile, 388: Error at src/reconcile/rec_write.c:388: "__rec_write_wrapup(session, r)" failed: Invalid argument
          [1773903939:624301][16675:0xffff6f9cdbc0], t, file:T00002.wt_stable, WT_SESSION.rollback_transaction: [WT_VERB_DEFAULT][ERROR]: __reconcile, 442: reconciliation failed after building the disk image: Invalid argument
          [1773903939:624304][16675:0xffff6f9cdbc0], t, file:T00002.wt_stable, WT_SESSION.rollback_transaction: [WT_VERB_DEFAULT][ERROR]: __reconcile, 442: the process must exit and restart: WT_PANIC: WiredTiger library panic
          [1773903939:624307][16675:0xffff6f9cdbc0], t, file:T00002.wt_stable, WT_SESSION.rollback_transaction: [WT_VERB_DEFAULT][ERROR]: __wt_abort, 29: aborting WiredTiger library
      

      logs

      format-stress-test-disagg-leader-1 task_log

      Logs:

      test/format run configuration highlights
      

      logs

      format-stress-test-disagg-leader-1 task_log

      Logs:

      #0  0x0000ffff8fec4454 in __pthread_kill_implementation () from /lib64/libc.so.6
      #0  0x0000ffff8fec4454 in __pthread_kill_implementation () from /lib64/libc.so.6
      #1  0x0000ffff8fe7b320 [PAC] in raise () from /lib64/libc.so.6
      #2  0x0000ffff8fe62224 [PAC] in abort () from /lib64/libc.so.6
      #3  0x0000ffff9023a8e0 [PAC] in __wt_abort (session=session@entry=0x53233fc9ed30) at /data/mci/8aeea9137f73ddff3b25c01c154f005a/wiredtiger/src/os_common/os_abort.c:32
      #4  0x0000ffff902df490 in __wt_panic_func (session=session@entry=0x53233fc9ed30, error=error@entry=22, func=func@entry=0xffff903dc268 <__PRETTY_FUNCTION__.63> "__reconcile", line=line@entry=442, category=category@entry=WT_VERB_DEFAULT, fmt=fmt@entry=0xffff90393ad0 "reconciliation failed after building the disk image") at /data/mci/8aeea9137f73ddff3b25c01c154f005a/wiredtiger/src/support/err.c:611
      #5  0x0000ffff90286604 in __reconcile (session=session@entry=0x53233fc9ed30, ref=ref@entry=0x53237a7c0f00, salvage=salvage@entry=0x0, flags=flags@entry=2728, page_lockedp=page_lockedp@entry=0xffff6f9cba4f) at /data/mci/8aeea9137f73ddff3b25c01c154f005a/wiredtiger/src/reconcile/rec_write.c:442
      #6  0x0000ffff902850d0 in __wt_reconcile (session=session@entry=0x53233fc9ed30, ref=ref@entry=0x53237a7c0f00, salvage=salvage@entry=0x0, flags=flags@entry=2728) at /data/mci/8aeea9137f73ddff3b25c01c154f005a/wiredtiger/src/reconcile/rec_write.c:127
      #7  0x0000ffff901f56fc in __evict_reconcile (session=session@entry=0x53233fc9ed30, ref=ref@entry=0x53237a7c0f00, evict_flags=evict_flags@entry=0) at /data/mci/8aeea9137f73ddff3b25c01c154f005a/wiredtiger/src/evict/evict_page.c:1271
      #8  0x0000ffff901f7988 in __wt_evict (session=session@entry=0x53233fc9ed30, ref=0x53237a7c0f00, previous_state=3 '\003', flags=flags@entry=0) at /data/mci/8aeea9137f73ddff3b25c01c154f005a/wiredtiger/src/evict/evict_page.c:434
      #9  0x0000ffff901eaaf8 in __evict_page (session=session@entry=0x53233fc9ed30, is_server=is_server@entry=false) at /data/mci/8aeea9137f73ddff3b25c01c154f005a/wiredtiger/src/evict/evict_lru.c:3093
      #10 0x0000ffff901ec014 in __wti_evict_app_assist_worker (session=session@entry=0x53233fc9ed30, busy=<optimized out>, busy@entry=false, readonly=readonly@entry=false, interruptible=interruptible@entry=true) at /data/mci/8aeea9137f73ddff3b25c01c154f005a/wiredtiger/src/evict/evict_lru.c:3224
      #11 0x0000ffff9030afb0 in __wt_evict_app_assist_worker_check (session=session@entry=0x53233fc9ed30, busy=busy@entry=false, readonly=readonly@entry=false, interruptible=interruptible@entry=true, didworkp=didworkp@entry=0x0) at /data/mci/8aeea9137f73ddff3b25c01c154f005a/wiredtiger/src/include/../evict/evict_inline.h:960
      #12 0x0000ffff9030f9d4 in __wt_txn_rollback (session=session@entry=0x53233fc9ed30, cfg=cfg@entry=0xffff6f9ccde8, api_call=api_call@entry=true) at /data/mci/8aeea9137f73ddff3b25c01c154f005a/wiredtiger/src/txn/txn.c:2172
      #13 0x0000ffff902ba924 in __session_rollback_transaction (wt_session=0x53233fc9ed30, config=<optimized out>) at /data/mci/8aeea9137f73ddff3b25c01c154f005a/wiredtiger/src/session/session_api.c:2044
      #14 0x00000000004163b8 in rollback_transaction (tinfo=tinfo@entry=0x53233b991180, prepared=prepared@entry=false) at /data/mci/8aeea9137f73ddff3b25c01c154f005a/wiredtiger/test/format/ops.c:660
      #15 0x0000000000417250 in ops (arg=0x53233b991180) at /data/mci/8aeea9137f73ddff3b25c01c154f005a/wiredtiger/test/format/ops.c:1410
      #16 0x0000ffff8fec2834 in start_thread () from /lib64/libc.so.6
      #17 0x0000ffff8fe66e5c [PAC] in thread_start () from /lib64/libc.so.6
      

      logs

      Repro Artifacts:

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            xgen-buildbaron-user
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: