Loading...

XML

Word

Printable

JSON

Type: Build Failure
Resolution: Unresolved
Priority: Major - P3
Fix Version/s: None
Affects Version/s: None
Component/s: None
Labels:
- BB-Tools

Assigned Teams:

Storage Engines - Transactions
Sprint:
None
Story Points:
None
Evergreen Project:
- wiredtiger
Failure Type:
- failed
Failing Buildvariants:
- amazon2023-armv9-release-nonstandalone
Failing Tasks:
- format-stress-test-1
First Failing Revision:
- 39f3858d49092565e4a4996c22b6a2be1eeddc54(wiredtiger)
Linked BFG List:
https://buildbaron.corp.mongodb.com/ui/#/bf/WT-16266
Count of Linked BFGs (Last 30 days):
1

format-stress-test-1 on amazon2023-armv9-release-nonstandalone

Host: i-043265b04d0e03d85
Project: wiredtiger
Commit: 39f3858d
Please refer to BF(G) Playbook for instructions on handling BF and BFG tickets as well as Auto-Resolution Rules

Task Logs:

task format-stress-test-1

format-stress-test-1 task_log

Logs:

    [1765433449:031395][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00002.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433519:534927][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433545:871906][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00003.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433572:186144][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433597:394803][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00002.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433622:339496][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00002.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433647:249849][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00003.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433673:607581][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433698:662375][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433723:585032][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433747:973294][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433772:898856][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00003.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433797:248323][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00002.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433822:090713][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433847:025446][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433873:597403][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433899:015114][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433923:435893][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00002.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433948:625104][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00003.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433974:168258][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00003.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765433999:923183][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00002.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434026:951173][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434053:263700][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434078:408485][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00003.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434106:118618][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434130:445073][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00002.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434155:474857][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00003.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434180:191225][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00003.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434207:673045][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00002.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434232:648354][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00003.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434257:543868][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434284:807654][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00002.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434310:205390][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00003.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434341:005107][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434365:568875][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434390:636655][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434416:635581][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00003.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434443:028870][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00003.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434467:464803][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00001.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434517:795551][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00002.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434548:999376][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00002.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434573:765974][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00002.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434599:678235][56981:0xffff8458dbc0], t, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __compact_worker, 344: Compaction halted at data handle file:T00002.wt by eviction pressure. Returning EBUSY.: Device or resource busy
    [1765434625:740614][56981:0xffff8458dbc0], t, WT_SESSION.compac

format-stress-test-1 task_log

Logs:

    format run more than 15 minutes past the maximum time

format-stress-test-1 task_log

Logs:

    format run dumping cache and transaction state, then aborting the process

format-stress-test-1 task_log

Logs:

    0x101abfe92000:transaction state dump
    0x101abfe92000:current ID: 205157
    0x101abfe92000:last running ID: 205157
    0x101abfe92000:metadata_pinned ID: 205156
    0x101abfe92000:oldest ID: 205157
    0x101abfe92000:durable timestamp: (0, 3801661)
    0x101abfe92000:oldest timestamp: (0, 3800575)
    0x101abfe92000:pinned timestamp: (0, 3800575)
    0x101abfe92000:stable timestamp: (0, 3800575)
    0x101abfe92000:has_durable_timestamp: yes
    0x101abfe92000:has_oldest_timestamp: yes
    0x101abfe92000:has_pinned_timestamp: yes
    0x101abfe92000:has_stable_timestamp: yes
    0x101abfe92000:oldest_is_pinned: yes
    0x101abfe92000:stable_is_pinned: yes
    0x101abfe92000:checkpoint running: yes
    0x101abfe92000:checkpoint generation: 125
    0x101abfe92000:checkpoint pinned ID: 205156
    0x101abfe92000:checkpoint txn ID: 205156
    0x101abfe92000:session count: 46
    0x101abfe92000:Transaction state of active sessions:
    0x101abfe92000:=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    0x101abfe92000:cache dump
    0x101abfe92000:cache full: no
    0x101abfe92000:cache clean check: no (21.421%)
    0x101abfe92000:cache dirty check: yes (20.026%)
    0x101abfe92000:cache updates check: no (1.693%)
    0x101abfe92000:file:T00003.wt(<live>):
    0x101abfe92000:internal: 1 pages, 79.46 KB, 1/0 clean/dirty pages, 79.46/0.00 clean / dirty KB, 79.46 KB max page, 0.00 KB max dirty page
    0x101abfe92000:leaf: 548 pages, 105487.33 KB, 37/511 clean/dirty pages, 6601.91 /98885.42 /7623.87 clean/dirty/updates KB, 274.19 KB max page, 274.19 KB max dirty page
    0x101abfe92000:file:T00002.wt(<live>):
    0x101abfe92000:internal: 1 pages, 50.45 KB, 1/0 clean/dirty pages, 50.45/0.00 clean / dirty KB, 50.45 KB max page, 0.00 KB max dirty page
    0x101abfe92000:leaf: 540 pages, 150036.78 KB, 69/471 clean/dirty pages, 17644.01 /132392.77 /10891.52 clean/dirty/updates KB, 362.89 KB max page, 362.89 KB max dirty page
    0x101abfe92000:file:T00001.wt(<live>):
    0x101abfe92000:internal: 7 pages, 30.67 KB, 5/2 clean/dirty pages, 25.58/5.09 clean / dirty KB, 5.58 KB max page, 4.17 KB max dirty page
    0x101abfe92000:leaf: 324 pages, 160254.05 KB, 6/318 clean/dirty pages, 2695.83 /157558.21 /14353.62 clean/dirty/updates KB, 609.48 KB max page, 609.48 KB max dirty page
    0x101abfe92000:file:WiredTigerHS.wt(<live>):
    0x101abfe92000:internal: 1 pages, 0.49 KB, 1/0 clean/dirty pages, 0.49/0.00 clean / dirty KB, 0.49 KB max page, 0.00 KB max dirty page
    0x101abfe92000:leaf: 0 pages
    0x101abfe92000:file:WiredTiger.wt(<live>):
    0x101abfe92000:internal: 1 pages, 0.54 KB, 1/0 clean/dirty pages, 0.54/0.00 clean / dirty KB, 0.54 KB max page, 0.00 KB max dirty page
    0x101abfe92000:leaf: 1 pages, 24.16 KB, 0/1 clean/dirty pages, 0.00 /24.16 /15.48 clean/dirty/updates KB, 24.16 KB max page, 24.16 KB max dirty page
    0x101abfe92000:cache dump: total found: 438.71 MB vs tracked inuse 406.21 MB
    0x101abfe92000:total dirty bytes: 379.75 MB vs tracked dirty 379.75 MB

format-stress-test-1 task_log

Logs:

    WiredTiger Error: __wt_abort, 29: aborting WiredTiger library

format-stress-test-1 task_log

Logs:

test/format run configuration highlights

format-stress-test-1 task_log

Logs:

#0  0x0000ffff9d3bd7b4 in __pthread_kill_implementation () from /lib64/libc.so.6
#0  0x0000ffff9d3bd7b4 in __pthread_kill_implementation () from /lib64/libc.so.6
#1  0x0000ffff9d3743a0 [PAC] in raise () from /lib64/libc.so.6
#2  0x0000ffff9d360264 [PAC] in abort () from /lib64/libc.so.6
#3  0x0000ffff9d64c9b8 [PAC] in __wt_abort (session=session@entry=0x0) at /data/mci/d80e5bb5a6b6a376e6c8117f869292b6/wiredtiger/src/os_common/os_abort.c:32
#4  0x0000000000415968 in operations (ops_seconds=<optimized out>, ops_seconds@entry=515, run_current=run_current@entry=1, run_total=run_total@entry=3) at /data/mci/d80e5bb5a6b6a376e6c8117f869292b6/wiredtiger/test/format/ops.c:461
#5  0x00000000004032b4 in main (argc=<optimized out>, argv=<optimized out>) at /data/mci/d80e5bb5a6b6a376e6c8117f869292b6/wiredtiger/test/format/t.c:394

Repro Artifacts:

CONFIG

Assignee:: [DO NOT USE] Backlog - Storage Engines Team
Reporter:: xgen-buildbaron-user
Votes:: 0 Vote for this issue
Watchers:: 2 Start watching this issue

Created:: Dec 12 2025 04:28:20 AM UTC
Updated:: Dec 14 2025 02:18:07 PM UTC