Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-13218

Panic in __tiered_server (test_tiered16) on RHEL8-zeries (v7.0)

    • Type: Icon: Build Failure Build Failure
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • StorEng - Defined Pipeline

      unit-test on rhel8-zseries

      Host: rhel83-z-7.dallasisv.build.10gen.cc
      Project: wiredtiger-mongo-v7.0
      Commit: 0d882da7

      Task Logs:

      unit-test task_log

      Logs:

      [pid:2572575]: ERROR: test_tiered16.test_tiered16.test_remove_shared(dir_store)
      ----------------------------------------------------------------------
      [pid:2572575]: testtools.testresult.real._StringException: lost connection during test 'test_tiered16.test_tiered16.test_remove_shared(dir_store)'
      ----------------------------------------------------------------------
      Ran 11076 tests in 5944.428s
      

      logs

      unit-test task_log

      Logs:

      #0  0x000003ff94bbea74 in raise () from /lib64/libc.so.6
      #0  0x000003ff94bbea74 in raise () from /lib64/libc.so.6
      #1  0x000003ff94ba3098 in abort () from /lib64/libc.so.6
      #2  0x000003ff93d58564 in __wt_abort (session=session@entry=0x3e609a18) at /data/mci/cede852400172aeff2ecd6e8d0363820/wiredtiger/src/os_common/os_abort.c:30
      #3  0x000003ff93de788c in __wt_panic_func (session=session@entry=0x3e609a18, error=<optimized out>, func=func@entry=0x3ff93e3f23c <__PRETTY_FUNCTION__.7> "__tiered_server", line=line@entry=411, category=category@entry=WT_VERB_DEFAULT, fmt=0x3ff93e3f16a "storage server error from %s") at /data/mci/cede852400172aeff2ecd6e8d0363820/wiredtiger/src/support/err.c:570
      #4  0x000003ff93c99680 in __tiered_server (arg=0x3e609a18) at /data/mci/cede852400172aeff2ecd6e8d0363820/wiredtiger/src/conn/conn_tiered.c:411
      #5  0x000003ff94f880b6 in start_thread () from /lib64/libpthread.so.0
      #6  0x000003ff94c7c75e in thread_start () from /lib64/libc.so.6
      

      logs

      unit-test task_log

      Logs:

      [1719634910:370446][2572579:0x3ff95976b30], test_tiered16.test_tiered16.test_remove_shared(dir_store), WT_SESSION.drop: [WT_VERB_DEFAULT][ERROR]: __drop_tiered, 196: drop for tiered storage object must configure removal of underlying files if forced removal of shared objects is enabled: Invalid argument
      [1719634910:370792][2572579:0x3ff95976b30], test_tiered16.test_tiered16.test_remove_shared(dir_store), WT_SESSION.drop: [WT_VERB_DEFAULT][ERROR]: __drop_tiered, 196: drop for tiered storage object must configure removal of underlying files if forced removal of shared objects is enabled: Invalid argument
      [1719634910:429238][2572579:0x3ff895fa910], test_tiered16.test_tiered16.test_remove_shared(dir_store), tiered-server: [WT_VERB_DEFAULT][ERROR]: __config_err, 18: Error parsing '' at offset 1088: Unexpected character: Invalid argument
      [1719634910:429272][2572579:0x3ff895fa910], test_tiered16.test_tiered16.test_remove_shared(dir_store), tiered-server: [WT_VERB_DEFAULT][ERROR]: __tiered_server, 411: storage server error from tier_storage_finish: Invalid argument
      [1719634910:429280][2572579:0x3ff895fa910], test_tiered16.test_tiered16.test_remove_shared(dir_store), tiered-server: [WT_VERB_DEFAULT][ERROR]: __tiered_server, 411: the process must exit and restart: WT_PANIC: WiredTiger library panic
      [1719634910:429298][2572579:0x3ff895fa910], test_tiered16.test_tiered16.test_remove_shared(dir_store), tiered-server: [WT_VERB_DEFAULT][ERROR]: __wt_abort, 28: aborting WiredTiger library
      

      logs

      unit-test task_log

      Logs:

      [1719634412:361169][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99032, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361177][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99033, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361183][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99034, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361189][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99035, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361196][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99036, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361201][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99037, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361208][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99038, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361214][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99039, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361224][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99040, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361230][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99041, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361236][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99042, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361242][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99043, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361248][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99044, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361254][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99045, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361260][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99046, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361265][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99047, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361271][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99048, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361277][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99049, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361282][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99050, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361291][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99051, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361297][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99052, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361302][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99053, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361308][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99054, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361313][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99055, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361319][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99056, txnid_not_visible=false, stable_timestamp=(0, 990) < durable_timestamp=(0, 1000): true, prepare_state=WT_PREPARE_INIT
      [1719634412:361324][2572577:0x3ff95976b30], test_rollback_to_stable22.test_rollback_to_stable22.test_rollback_to_stable, file:rollback_to_stable22_0.wt, txn rollback_to_stable: [WT_VERB_RTS][DEBUG_1]: [UPDATE_ABORT] rollback to stable aborting update with txnid=99057, txnid_not_visible=false, stable_timestamp=(0, 990) < dur
      

      logs

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Xgen-BuildBaron-User xgen-buildbaron-user
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: