-
Type:
Build Failure
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Storage Engines, Storage Engines - Transactions
-
None
-
None
-
1
format-stress-sanitizer-ppc-test-2 on rhel8-ppc
Host: rhel8-p9-small-8.power.devprod.mongodb.com
Project: wiredtiger
Commit: 7c8f7ec5
Please refer to BF(G) Playbook for instructions on handling BF and BFG tickets as well as Auto-Resolution Rules
Task Logs:
format-stress-sanitizer-ppc-test-2 task_log
Logs:
[1750177458:113595][1031992:0x7fff3bfff170], t, eviction-server: [WT_VERB_DEFAULT][ERROR]: __evict_server, 539: Cache stuck for too long, giving up: Connection timed out
format-stress-sanitizer-ppc-test-2 task_log
Logs:
0x7fff62c7fb50:transaction state dump
0x7fff62c7fb50:current ID: 20
0x7fff62c7fb50:last running ID: 20
0x7fff62c7fb50:metadata_pinned ID: 20
0x7fff62c7fb50:oldest ID: 20
0x7fff62c7fb50:durable timestamp: (0, 1502493)
0x7fff62c7fb50:oldest timestamp: (0, 1502493)
0x7fff62c7fb50:pinned timestamp: (0, 1502493)
0x7fff62c7fb50:stable timestamp: (0, 1502493)
0x7fff62c7fb50:has_durable_timestamp: yes
0x7fff62c7fb50:has_oldest_timestamp: yes
0x7fff62c7fb50:has_pinned_timestamp: yes
0x7fff62c7fb50:has_stable_timestamp: yes
0x7fff62c7fb50:oldest_is_pinned: yes
0x7fff62c7fb50:stable_is_pinned: no
0x7fff62c7fb50:checkpoint running: no
0x7fff62c7fb50:checkpoint generation: 3
0x7fff62c7fb50:checkpoint pinned ID: 0
0x7fff62c7fb50:checkpoint txn ID: 0
0x7fff62c7fb50:session count: 29
0x7fff62c7fb50:Transaction state of active sessions:
0x7fff62c7fb50:=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
0x7fff62c7fb50:cache dump
0x7fff62c7fb50:cache full: no
0x7fff62c7fb50:cache clean check: no (24.161%)
0x7fff62c7fb50:cache dirty check: yes (3.771%)
0x7fff62c7fb50:cache updates check: no (3.763%)
0x7fff62c7fb50:file:T00003.wt(<live>):
0x7fff62c7fb50:internal: 1 pages, 280.10 KB, 1/0 clean/dirty pages, 280.10/0.00 clean / dirty KB, 280.10 KB max page, 0.00 KB max dirty page
0x7fff62c7fb50:leaf: 41 pages, 4857.40 KB, 41/0 clean/dirty pages, 4857.40 /0.00 /0.00 clean/dirty/updates KB, 127.04 KB max page, 0.00 KB max dirty page
0x7fff62c7fb50:file:T00002.wt(<live>):
0x7fff62c7fb50:internal: 649 pages, 14481.73 KB, 647/2 clean/dirty pages, 14474.20/7.54 clean / dirty KB, 30.27 KB max page, 4.31 KB max dirty page
0x7fff62c7fb50:leaf: 236 pages, 1795.87 KB, 236/0 clean/dirty pages, 1795.87 /0.00 /0.00 clean/dirty/updates KB, 14.13 KB max page, 0.00 KB max dirty page
0x7fff62c7fb50:file:T00001.wt(<live>):
0x7fff62c7fb50:internal: 3 pages, 41.49 KB, 2/1 clean/dirty pages, 20.25/21.24 clean / dirty KB, 21.24 KB max page, 21.24 KB max dirty page
0x7fff62c7fb50:leaf: 0 pages
0x7fff62c7fb50:file:WiredTigerHS.wt(<live>):
0x7fff62c7fb50:internal: 15 pages, 557.44 KB, 8/7 clean/dirty pages, 364.33/193.10 clean / dirty KB, 161.21 KB max page, 92.12 KB max dirty page
0x7fff62c7fb50:leaf: 374 pages, 1184.84 KB, 374/0 clean/dirty pages, 1184.84 /0.00 /0.00 clean/dirty/updates KB, 22.88 KB max page, 0.00 KB max dirty page
0x7fff62c7fb50:file:WiredTiger.wt(<live>):
0x7fff62c7fb50:internal: 1 pages, 0.77 KB, 1/0 clean/dirty pages, 0.77/0.00 clean / dirty KB, 0.77 KB max page, 0.00 KB max dirty page
0x7fff62c7fb50:leaf: 1 pages, 4290.40 KB, 0/1 clean/dirty pages, 0.00 /4290.40 /4281.65 clean/dirty/updates KB, 4290.40 KB max page, 4290.40 KB max dirty page
0x7fff62c7fb50:cache dump: total found: 28.99 MB vs tracked inuse 26.85 MB
0x7fff62c7fb50:total dirty bytes: 4.41 MB vs tracked dirty 4.41 MB
format-stress-sanitizer-ppc-test-2 task_log
Logs:
[1750177458:120352][1031992:0x7fff3bfff170], t, eviction-server: [WT_VERB_DEFAULT][ERROR]: __evict_thread_run, 357: eviction thread error: Connection timed out
[1750177458:120391][1031992:0x7fff3bfff170], t, eviction-server: [WT_VERB_DEFAULT][ERROR]: __evict_thread_run, 357: the process must exit and restart: WT_PANIC: WiredTiger library panic
[1750177458:120405][1031992:0x7fff3bfff170], t, eviction-server: [WT_VERB_DEFAULT][ERROR]: __wt_abort, 29: aborting WiredTiger library
format-stress-sanitizer-ppc-test-2 task_log
Logs:
#0 0x00007fffa13e96f8 in raise () from /lib64/glibc-hwcaps/power9/libc-2.28.so #0 0x00007fffa13e96f8 in raise () from /lib64/glibc-hwcaps/power9/libc-2.28.so #1 0x00007fffa13c3ff4 in abort () from /lib64/glibc-hwcaps/power9/libc-2.28.so #2 0x00007fffa18af0c8 in __wt_abort (session=<optimized out>) at /data/mci/350ce1e2e351f7b97022f5a3b67dc84b/wiredtiger/src/os_common/os_abort.c:31 #3 0x00007fffa19666b8 in __wt_panic_func (session=0x7fff62c7fb50, error=<optimized out>, func=0x7fffa19ff330 <__PRETTY_FUNCTION__.21257> "__evict_thread_run", line=<optimized out>, category=<optimized out>, fmt=<optimized out>) at /data/mci/350ce1e2e351f7b97022f5a3b67dc84b/wiredtiger/src/support/err.c:570 #4 0x00007fffa185c75c in __evict_thread_run (session=0x7fff62c7fb50, thread=0x7fff60590f10) at /data/mci/350ce1e2e351f7b97022f5a3b67dc84b/wiredtiger/src/evict/evict_lru.c:357 #5 0x00007fffa198ced8 in __thread_run (arg=0x7fff60590f10) at /data/mci/350ce1e2e351f7b97022f5a3b67dc84b/wiredtiger/src/support/thread_group.c:31 #6 0x00007fffa1619678 in start_thread () from /lib64/glibc-hwcaps/power9/libpthread-2.28.so #7 0x00007fffa14d8938 in clone () from /lib64/glibc-hwcaps/power9/libc-2.28.so
Repro Artifacts: