-
Type:
Build Failure
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Storage Engines, Storage Engines - Transactions
-
SE Transactions - 2025-09-26
-
5
-
8
format-abort-recovery-stress-test on ubuntu2004-arm64-nonstandalone
Host: i-068e26792ff873e82
Project: wiredtiger
Commit: ee5b8cd4
Please refer to BF(G) Playbook for instructions on handling BF and BFG tickets as well as Auto-Resolution Rules
Task Logs:
format-abort-recovery-stress-test task_log
Logs:
./format.sh: line 707: 73165 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 73084 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 73223 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 74041 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 73459 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 72938 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 72960 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 73413 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 73658 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 73756 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 73945 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 707: 73299 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 73565 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 73844 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 77988 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 78231 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 78149 Segmentation fault nohup setsid $cmd > $log 2>&1
format-abort-recovery-stress-test task_log
Logs:
[1757053641:348053][72899:0xffff9822bc40], t, file:WiredTigerHS.wt, eviction-server: [WT_VERB_DEFAULT][ERROR]: __evict_server, 541: Cache stuck for too long, giving up: Connection timed out
format-abort-recovery-stress-test task_log
Logs:
0x52b17f598f40:transaction state dump 0x52b17f598f40:current ID: 733071 0x52b17f598f40:last running ID: 733071 0x52b17f598f40:metadata_pinned ID: 733070 0x52b17f598f40:oldest ID: 733071 0x52b17f598f40:durable timestamp: (0, 20559078) 0x52b17f598f40:oldest timestamp: (0, 20333663) 0x52b17f598f40:pinned timestamp: (0, 20333663) 0x52b17f598f40:stable timestamp: (0, 20333663) 0x52b17f598f40:has_durable_timestamp: yes 0x52b17f598f40:has_oldest_timestamp: yes 0x52b17f598f40:has_pinned_timestamp: yes 0x52b17f598f40:has_stable_timestamp: yes 0x52b17f598f40:oldest_is_pinned: yes 0x52b17f598f40:stable_is_pinned: yes 0x52b17f598f40:checkpoint running: yes 0x52b17f598f40:checkpoint generation: 9 0x52b17f598f40:checkpoint pinned ID: 733070 0x52b17f598f40:checkpoint txn ID: 733070 0x52b17f598f40:session count: 26 0x52b17f598f40:Transaction state of active sessions: 0x52b17f598f40:=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= 0x52b17f598f40:cache dump 0x52b17f598f40:cache full: no 0x52b17f598f40:cache clean check: no (39.480%) 0x52b17f598f40:cache dirty check: yes (27.234%) 0x52b17f598f40:cache updates check: no (7.224%) 0x52b17f598f40:file:T00002.wt(<live>): 0x52b17f598f40:internal: 99 pages, 27105.72 KB, 99/0 clean/dirty pages, 27105.72/0.00 clean / dirty KB, 369.92 KB max page, 0.00 KB max dirty page 0x52b17f598f40:leaf: 143096 pages, 414558.80 KB, 78148/64948 clean/dirty pages, 190211.01 /224347.79 /56505.29 clean/dirty/updates KB, 178.63 KB max page, 178.63 KB max dirty page 0x52b17f598f40:file:T00001.wt(<live>): 0x52b17f598f40:internal: 1 pages, 37.19 KB, 1/0 clean/dirty pages, 37.19/0.00 clean / dirty KB, 37.19 KB max page, 0.00 KB max dirty page 0x52b17f598f40:leaf: 212 pages, 172194.19 KB, 1/211 clean/dirty pages, 378.12 /171816.08 /56177.19 clean/dirty/updates KB, 7852.47 KB max page, 7852.47 KB max dirty page 0x52b17f598f40:file:T00005.wt(<live>): 0x52b17f598f40:internal: 1 pages, 226.31 KB, 1/0 clean/dirty pages, 226.31/0.00 clean / dirty KB, 226.31 KB max page, 0.00 KB max dirty page 0x52b17f598f40:leaf: 1315 pages, 362494.88 KB, 0/1315 clean/dirty pages, 0.00 /362494.88 /72061.88 clean/dirty/updates KB, 623.29 KB max page, 623.29 KB max dirty page 0x52b17f598f40:file:T00004.wt(<live>): 0x52b17f598f40:internal: 466 pages, 28075.91 KB, 466/0 clean/dirty pages, 28075.91/0.00 clean / dirty KB, 195.79 KB max page, 0.00 KB max dirty page 0x52b17f598f40:leaf: 155181 pages, 352920.67 KB, 112888/42293 clean/dirty pages, 227796.22 /125124.45 /39451.56 clean/dirty/updates KB, 5.67 KB max page, 5.67 KB max dirty page 0x52b17f598f40:file:F00003(<live>): 0x52b17f598f40:internal: 7 pages, 93.25 KB, 0/7 clean/dirty pages, 0.00/93.25 clean / dirty KB, 16.75 KB max page, 16.75 KB max dirty page 0x52b17f598f40:leaf: 510 pages, 171769.70 KB, 2/508 clean/dirty pages, 508.26 /171261.43 /55659.93 clean/dirty/updates KB, 7401.15 KB max page, 7401.15 KB max dirty page 0x52b17f598f40:file:WiredTigerHS.wt(<live>): 0x52b17f598f40:internal: 1 pages, 0.78 KB, 1/0 clean/dirty pages, 0.78/0.00 clean / dirty KB, 0.78 KB max page, 0.00 KB max dirty page 0x52b17f598f40:leaf: 0 pages 0x52b17f598f40:file:WiredTiger.wt(<live>): 0x52b17f598f40:internal: 1 pages, 0.77 KB, 1/0 clean/dirty pages, 0.77/0.00 clean / dirty KB, 0.77 KB max page, 0.00 KB max dirty page 0x52b17f598f40:leaf: 1 pages, 30.93 KB, 0/1 clean/dirty pages, 0.00 /30.93 /20.29 clean/dirty/updates KB, 30.93 KB max page, 30.93 KB max dirty page 0x52b17f598f40:cache dump: total found: 1613.15 MB vs tracked inuse 1493.66 MB 0x52b17f598f40:total dirty bytes: 1030.44 MB vs tracked dirty 1030.44 MB
format-abort-recovery-stress-test task_log
Logs:
[1757053641:529727][72899:0xffff9822bc40], t, file:WiredTigerHS.wt, eviction-server: [WT_VERB_DEFAULT][ERROR]: __evict_thread_run, 358: eviction thread error: Connection timed out [1757053641:529739][72899:0xffff9822bc40], t, file:WiredTigerHS.wt, eviction-server: [WT_VERB_DEFAULT][ERROR]: __evict_thread_run, 358: the process must exit and restart: WT_PANIC: WiredTiger library panic [1757053641:529744][72899:0xffff9822bc40], t, file:WiredTigerHS.wt, eviction-server: [WT_VERB_DEFAULT][ERROR]: __wt_abort, 29: aborting WiredTiger library
format-abort-recovery-stress-test task_log
Logs:
test/format run configuration highlights
format-abort-recovery-stress-test task_log
Logs:
./format.sh: line 707: 78680 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 707: 78557 Segmentation fault nohup setsid $cmd > $log 2>&1 ./format.sh: line 514: 77704 Segmentation fault nohup setsid $cmd > $log 2>&1
format-abort-recovery-stress-test task_log
Logs:
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x0000ffff98dbbaac in __GI_abort () at abort.c:79 #2 0x0000ffff99132410 in __wt_abort (session=session@entry=0x52b17f598f40) at /data/mci/fae4a1f1a7cfd5ba622583f19f828858/wiredtiger/src/os_common/os_abort.c:31 #3 0x0000ffff991b82f8 in __wt_panic_func (session=session@entry=0x52b17f598f40, error=error@entry=110, func=func@entry=0xffff99244c78 <__PRETTY_FUNCTION__.21> "__evict_thread_run", line=line@entry=358, category=category@entry=WT_VERB_DEFAULT, fmt=fmt@entry=0xffff9920f3e8 "eviction thread error") at /data/mci/fae4a1f1a7cfd5ba622583f19f828858/wiredtiger/src/support/err.c:572 #4 0x0000ffff990fa730 in __evict_thread_run (session=0x52b17f598f40, thread=0x52b17fc0e910) at /data/mci/fae4a1f1a7cfd5ba622583f19f828858/wiredtiger/src/evict/evict_lru.c:358 #5 0x0000ffff991d0a88 in __thread_run (arg=0x52b17fc0e910) at /data/mci/fae4a1f1a7cfd5ba622583f19f828858/wiredtiger/src/support/thread_group.c:31 #6 0x0000ffff98f41624 in start_thread (arg=0xffff991d0a38 <__thread_run>) at pthread_create.c:477 #7 0x0000ffff98e6c66c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78
Repro Artifacts: