failed: format-abort-recovery-stress-test on ubuntu2004-arm64-nonstandalone [wiredtiger @ ee5b8cd4]

XMLWordPrintableJSON

    • Type: Build Failure
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None

      format-abort-recovery-stress-test on ubuntu2004-arm64-nonstandalone

      Host: i-068e26792ff873e82
      Project: wiredtiger
      Commit: ee5b8cd4
      Please refer to BF(G) Playbook for instructions on handling BF and BFG tickets as well as Auto-Resolution Rules

      Task Logs:

      format-abort-recovery-stress-test task_log

      Logs:

      ./format.sh: line 707: 73165 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 73084 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 73223 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 74041 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 73459 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 72938 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 72960 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 73413 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 73658 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 73756 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 73945 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 707: 73299 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 73565 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 73844 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 77988 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 78231 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 78149 Segmentation fault      nohup setsid $cmd > $log 2>&1
      

      logs

      format-abort-recovery-stress-test task_log

      Logs:

          [1757053641:348053][72899:0xffff9822bc40], t, file:WiredTigerHS.wt, eviction-server: [WT_VERB_DEFAULT][ERROR]: __evict_server, 541: Cache stuck for too long, giving up: Connection timed out
      
      

      logs

      format-abort-recovery-stress-test task_log

      Logs:

          0x52b17f598f40:transaction state dump
          0x52b17f598f40:current ID: 733071
          0x52b17f598f40:last running ID: 733071
          0x52b17f598f40:metadata_pinned ID: 733070
          0x52b17f598f40:oldest ID: 733071
          0x52b17f598f40:durable timestamp: (0, 20559078)
          0x52b17f598f40:oldest timestamp: (0, 20333663)
          0x52b17f598f40:pinned timestamp: (0, 20333663)
          0x52b17f598f40:stable timestamp: (0, 20333663)
          0x52b17f598f40:has_durable_timestamp: yes
          0x52b17f598f40:has_oldest_timestamp: yes
          0x52b17f598f40:has_pinned_timestamp: yes
          0x52b17f598f40:has_stable_timestamp: yes
          0x52b17f598f40:oldest_is_pinned: yes
          0x52b17f598f40:stable_is_pinned: yes
          0x52b17f598f40:checkpoint running: yes
          0x52b17f598f40:checkpoint generation: 9
          0x52b17f598f40:checkpoint pinned ID: 733070
          0x52b17f598f40:checkpoint txn ID: 733070
          0x52b17f598f40:session count: 26
          0x52b17f598f40:Transaction state of active sessions:
          0x52b17f598f40:=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
          0x52b17f598f40:cache dump
          0x52b17f598f40:cache full: no
          0x52b17f598f40:cache clean check: no (39.480%)
          0x52b17f598f40:cache dirty check: yes (27.234%)
          0x52b17f598f40:cache updates check: no (7.224%)
          0x52b17f598f40:file:T00002.wt(<live>):
          0x52b17f598f40:internal: 99 pages, 27105.72 KB, 99/0 clean/dirty pages, 27105.72/0.00 clean / dirty KB, 369.92 KB max page, 0.00 KB max dirty page
          0x52b17f598f40:leaf: 143096 pages, 414558.80 KB, 78148/64948 clean/dirty pages, 190211.01 /224347.79 /56505.29 clean/dirty/updates KB, 178.63 KB max page, 178.63 KB max dirty page
          0x52b17f598f40:file:T00001.wt(<live>):
          0x52b17f598f40:internal: 1 pages, 37.19 KB, 1/0 clean/dirty pages, 37.19/0.00 clean / dirty KB, 37.19 KB max page, 0.00 KB max dirty page
          0x52b17f598f40:leaf: 212 pages, 172194.19 KB, 1/211 clean/dirty pages, 378.12 /171816.08 /56177.19 clean/dirty/updates KB, 7852.47 KB max page, 7852.47 KB max dirty page
          0x52b17f598f40:file:T00005.wt(<live>):
          0x52b17f598f40:internal: 1 pages, 226.31 KB, 1/0 clean/dirty pages, 226.31/0.00 clean / dirty KB, 226.31 KB max page, 0.00 KB max dirty page
          0x52b17f598f40:leaf: 1315 pages, 362494.88 KB, 0/1315 clean/dirty pages, 0.00 /362494.88 /72061.88 clean/dirty/updates KB, 623.29 KB max page, 623.29 KB max dirty page
          0x52b17f598f40:file:T00004.wt(<live>):
          0x52b17f598f40:internal: 466 pages, 28075.91 KB, 466/0 clean/dirty pages, 28075.91/0.00 clean / dirty KB, 195.79 KB max page, 0.00 KB max dirty page
          0x52b17f598f40:leaf: 155181 pages, 352920.67 KB, 112888/42293 clean/dirty pages, 227796.22 /125124.45 /39451.56 clean/dirty/updates KB, 5.67 KB max page, 5.67 KB max dirty page
          0x52b17f598f40:file:F00003(<live>):
          0x52b17f598f40:internal: 7 pages, 93.25 KB, 0/7 clean/dirty pages, 0.00/93.25 clean / dirty KB, 16.75 KB max page, 16.75 KB max dirty page
          0x52b17f598f40:leaf: 510 pages, 171769.70 KB, 2/508 clean/dirty pages, 508.26 /171261.43 /55659.93 clean/dirty/updates KB, 7401.15 KB max page, 7401.15 KB max dirty page
          0x52b17f598f40:file:WiredTigerHS.wt(<live>):
          0x52b17f598f40:internal: 1 pages, 0.78 KB, 1/0 clean/dirty pages, 0.78/0.00 clean / dirty KB, 0.78 KB max page, 0.00 KB max dirty page
          0x52b17f598f40:leaf: 0 pages
          0x52b17f598f40:file:WiredTiger.wt(<live>):
          0x52b17f598f40:internal: 1 pages, 0.77 KB, 1/0 clean/dirty pages, 0.77/0.00 clean / dirty KB, 0.77 KB max page, 0.00 KB max dirty page
          0x52b17f598f40:leaf: 1 pages, 30.93 KB, 0/1 clean/dirty pages, 0.00 /30.93 /20.29 clean/dirty/updates KB, 30.93 KB max page, 30.93 KB max dirty page
          0x52b17f598f40:cache dump: total found: 1613.15 MB vs tracked inuse 1493.66 MB
          0x52b17f598f40:total dirty bytes: 1030.44 MB vs tracked dirty 1030.44 MB
      

      logs

      format-abort-recovery-stress-test task_log

      Logs:

          [1757053641:529727][72899:0xffff9822bc40], t, file:WiredTigerHS.wt, eviction-server: [WT_VERB_DEFAULT][ERROR]: __evict_thread_run, 358: eviction thread error: Connection timed out
          [1757053641:529739][72899:0xffff9822bc40], t, file:WiredTigerHS.wt, eviction-server: [WT_VERB_DEFAULT][ERROR]: __evict_thread_run, 358: the process must exit and restart: WT_PANIC: WiredTiger library panic
          [1757053641:529744][72899:0xffff9822bc40], t, file:WiredTigerHS.wt, eviction-server: [WT_VERB_DEFAULT][ERROR]: __wt_abort, 29: aborting WiredTiger library
      

      logs

      format-abort-recovery-stress-test task_log

      Logs:

      test/format run configuration highlights
      

      logs

      format-abort-recovery-stress-test task_log

      Logs:

      ./format.sh: line 707: 78680 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 707: 78557 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 514: 77704 Segmentation fault      nohup setsid $cmd > $log 2>&1
      

      logs

      format-abort-recovery-stress-test task_log

      Logs:

      #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
      #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
      #1  0x0000ffff98dbbaac in __GI_abort () at abort.c:79
      #2  0x0000ffff99132410 in __wt_abort (session=session@entry=0x52b17f598f40) at /data/mci/fae4a1f1a7cfd5ba622583f19f828858/wiredtiger/src/os_common/os_abort.c:31
      #3  0x0000ffff991b82f8 in __wt_panic_func (session=session@entry=0x52b17f598f40, error=error@entry=110, func=func@entry=0xffff99244c78 <__PRETTY_FUNCTION__.21> "__evict_thread_run", line=line@entry=358, category=category@entry=WT_VERB_DEFAULT, fmt=fmt@entry=0xffff9920f3e8 "eviction thread error") at /data/mci/fae4a1f1a7cfd5ba622583f19f828858/wiredtiger/src/support/err.c:572
      #4  0x0000ffff990fa730 in __evict_thread_run (session=0x52b17f598f40, thread=0x52b17fc0e910) at /data/mci/fae4a1f1a7cfd5ba622583f19f828858/wiredtiger/src/evict/evict_lru.c:358
      #5  0x0000ffff991d0a88 in __thread_run (arg=0x52b17fc0e910) at /data/mci/fae4a1f1a7cfd5ba622583f19f828858/wiredtiger/src/support/thread_group.c:31
      #6  0x0000ffff98f41624 in start_thread (arg=0xffff991d0a38 <__thread_run>) at pthread_create.c:477
      #7  0x0000ffff98e6c66c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78
      

      logs

      Repro Artifacts:

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            xgen-buildbaron-user
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: