Loading...

Type: Build Failure
Resolution: Fixed
Priority: Major - P3
Fix Version/s: WT11.2.0, 6.3.0-rc0
Affects Version/s: None
Component/s: None
Labels:

Sprint:
StorEng - Defined Pipeline
Story Points:
3
Evergreen Project:
- wiredtiger
Failure Type:
- failed
Failing Buildvariants:
- ubuntu2004-stress-tests-arm64
Failing Tasks:
- format-stress-test-3
First Failing Revision:
- c5e3a8a043ba2c59c6cb3884f4d729992e2e1185
Linked BFG List:
https://buildbaron.corp.mongodb.com/ui/#/bf/WT-9292
Count of Linked BFGs (Last 30 days):
0

format-stress-test-3 on ubuntu2004-stress-tests-arm64

Host: i-00078ad4b9bae9cf3
Project: wiredtiger
Commit: c5e3a8a0

Signature:

 [2022/05/05 12:49:54.597]     [1651754989:719417][112727:0xffff97fff1c0], t, file:T00004.wt, WT_CURSOR.next: [WT_VERB_DEFAULT][ERROR]: __wt_btcur_evict_reposition, 648: failed to reposition the cursor: WT_NOTFOUND: item not found
 [2022/05/05 12:49:54.597]     [1651754989:719443][112727:0xffff97fff1c0], t, file:T00004.wt, WT_CURSOR.next: [WT_VERB_DEFAULT][ERROR]: __wt_btcur_evict_reposition, 648: the process must exit and restart: WT_PANIC: WiredTiger library panic
 [2022/05/05 12:49:54.597]     [1651754989:719448][112727:0xffff97fff1c0], t, file:T00004.wt, WT_CURSOR.next: [WT_VERB_DEFAULT][ERROR]: __wt_abort, 28: aborting WiredTiger library

Task Logs:

task format-stress-test-3

format-stress-test-3 task_log

Logs:

cr_granularity=15734
    block_cache=0
    block_cache.cache_on_checkpoint=1
    block_cache.cache_on_writes=0
    block_cache.size=62
    btree.huffman_value=0
    cache=280
    cache.evict_max=5
    cache.minimum=20
    checkpoint=on
    checkpoint.log_size=187
    checkpoint.wait=70
    disk.data_extend=0
    disk.direct_io=0
    disk.encryption=none
    disk.mmap=1
    disk.mmap_all=0
    format.abort=0
    format.independent_thread_rng=1
    format.major_timeout=0
    import=0
    logging=0
    logging.compression=none
    logging.file_max=13614
    logging.prealloc=0
    logging.remove=0
    ops.alter=0
    ops.compaction=0
    ops.hs_cursor=0
    ops.prepare=0
    ops.random_cursor=1
    ops.salvage=0
    ops.verify=1
    quiet=1
    runs.in_memory=0
    runs.ops=0
    runs.rows=1000000
    runs.tables=5
    runs.threads=4
    runs.timer=6
    runs.type=row-store
    runs.verify_failure_dump=0
    statistics=0
    statistics.server=0
    stress.aggressive_sweep=0
    stress.checkpoint=0
    stress.checkpoint_reserved_txnid_delay=0
    stress.checkpoint_prepare=0
    stress.evict_reposition=0
    stress.failpoint_hs_delete_key_from_ts=0
    stress.hs_checkpoint_delay=0
    stress.hs_search=0
    stress.hs_sweep=0
    stress.split_1=0
    stress.split_2=0
    stress.split_3=0
    stress.split_4=0
    stress.split_5=1
    stress.split_6=0
    stress.split_7=0
    transaction.implicit=0
    transaction.timestamps=1
    wiredtiger.config=off
    wiredtiger.rwlock=1
    wiredtiger.leak_memory=0
    ############################################
    #  TABLE PARAMETERS: table 1
    ############################################
    table1.btree.compression=lz4
    table1.btree.dictionary=0
    table1.btree.internal_key_truncation=1
    table1.btree.internal_page_max=16
    table1.btree.key_max=50
    table1.btree.key_min=17
    table1.btree.leaf_page_max=10
    table1.btree.memory_page_max=9
    table1.btree.prefix_len=0
    table1.btree.prefix_compression=1
    table1.btree.prefix_compression_min=0
    table1.btree.reverse=0
    table1.btree.split_pct=85
    table1.btree.value_max=3781
    table1.btree.value_min=14
    table1.disk.checksum=uncompressed
    table1.disk.firstfit=0
    table1.ops.pct.delete=13
    table1.ops.pct.insert=25
    table1.ops.pct.modify=23
    table1.ops.pct.read=13
    table1.ops.pct.write=26
    table1.ops.truncate=1
    table1.runs.source=file
    ############################################
    #  TABLE PARAMETERS: table 2
    ############################################
    table2.btree.compression=lz4
    table2.btree.dictionary=0
    table2.btree.internal_key_truncation=1
    table2.btree.internal_page_max=16
    table2.btree.leaf_page_max=10
    table2.btree.memory_page_max=5
    table2.btree.repeat_data_pct=19
    table2.btree.split_pct=73
    table2.btree.value_max=3384
    table2.btree.value_min=19
    table2.disk.checksum=unencrypted
    table2.disk.firstfit=0
    table2.ops.pct.delete=1
    table2.ops.pct.insert=86
    table2.ops.pct.modify=2
    table2.ops.pct.read=11
    table2.ops.pct.write=0
    table2.ops.truncate=1
    table2.runs.source=table
    table2.runs.type=variable-length column-store
    ############################################
    #  TABLE PARAMETERS: table 3
    ############################################
    table3.btree.compression=snappy
    table3.btree.dictionary=0
    table3.btree.internal_key_truncation=1
    table3.btree.internal_page_max=9
    table3.btree.leaf_page_max=16
    table3.btree.memory_page_max=3
    table3.btree.repeat_data_pct=76
    table3.btree.split_pct=57
    table3.btree.value_max=1782
    table3.btree.value_min=0
    table3.disk.checksum=on
    table3.disk.firstfit=0
    table3.ops.pct.delete=1
    table3.ops.pct.insert=49
    table3.ops.pct.modify=38
    table3.ops.pct.read=5
    table3.ops.pct.write=7
    table3.ops.truncate=1
    table3.runs.source=file
    table3.runs.type=variable-length column-store
    ############################################
    #  TABLE PARAMETERS: table 4
    ############################################
    table4.btree.compression=none
    table4.btree.dictionary=0
    table4.btree.internal_key_truncation=1
    table4.btree.internal_page_max=10
    table4.btree.key_max=118
    table4.btree.key_min=31
    table4.btree.leaf_page_max=10
    table4.btree.memory_page_max=7
    table4.btree.prefix_len=0
    table4.btree.prefix_compression=0
    table4.btree.prefix_compression_min=6
    table4.btree.reverse=0
    table4.btree.split_pct=87
    table4.btree.value_max=1979
    table4.btree.value_min=0
    table4.disk.checksum=off
    table4.disk.firstfit=0
    table4.ops.pct.delete=52
    table4.ops.pct.insert=5
    table4.ops.pct.modify=0
    table4.ops.pct.read=31
    table4.ops.pct.write=12
    table4.ops.truncate=1
    table4.runs.source=table
    ############################################
    #  TABLE PARAMETERS: table 5
    ############################################
    table5.btree.compression=snappy
    table5.btree.dictionary=0
    table5.btree.internal_key_truncation=1
    table5.btree.internal_page_max=12
    table5.btree.key_max=113
    table5.btree.key_min=28
    table5.btree.leaf_page_max=13
    table5.btree.memory_page_max=4
    table5.btree.prefix_len=0
    table5.btree.prefix_compression=1
    table5.btree.prefix_compression_min=0
    table5.btree.reverse=0
    table5.btree.split_pct=64
    table5.btree.value_max=1035
    table5.btree.value_min=4
    table5.disk.checksum=on
    table5.disk.firstfit=1
    table5.ops.pct.delete=65
    table5.ops.pct.insert=0
    table5.ops.pct.modify=8
    table5.ops.pct.read=26
    table5.ops.pct.write=1
    table5.ops.truncate=1
    table5.runs.source=table
############################################
test/format run configuration highlights
############################################
############################################
format.sh: 121 successful jobs, 1 failed jobs
format.sh: starting job in /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.130 (Thu May  5 12:49:56 UTC 2022)
format.sh:  catchsegv ./t -c /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/CONFIG.stress -h /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.130   quiet=1
./format.sh: line 410: wait: pid 108347 is not a child of this shell
format.sh: job in /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.117 successfully completed
format.sh: 122 successful jobs, 1 failed jobs
format.sh: starting job in /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.131 (Thu May  5 12:54:46 UTC 2022)
format.sh:  catchsegv ./t -c /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/CONFIG.stress -h /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.131   quiet=1
./format.sh: line 410: wait: pid 112255 is not a child of this shell
format.sh: job in /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.126 successfully completed
format.sh: 123 successful jobs, 1 failed jobs
format.sh: starting job in /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.132 (Thu May  5 12:55:01 UTC 2022)
format.sh:  catchsegv ./t -c /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/CONFIG.stress -h /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.132   quiet=1
./format.sh: line 410: wait: pid 112855 is not a child of this shell
format.sh: job in /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.129 successfully completed
format.sh: 124 successful jobs, 1 failed jobs
format.sh: starting job in /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.133 (Thu May  5 12:56:29 UTC 2022)
format.sh:  catchsegv ./t -c /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/CONFIG.stress -h /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.133   quiet=1
./format.sh: line 410: wait: pid 112326 is not a child of this shell
format.sh: job in /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.127 successfully completed
format.sh: 125 successful jobs, 1 failed jobs
format.sh: starting job in /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.134 (Thu May  5 12:58:30 UTC 2022)
format.sh:  catchsegv ./t -c /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/CONFIG.stress -h /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.134   quiet=1
./format.sh: line 410: wait: pid 115173 is not a child of this shell
format.sh: job in /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.134 successfully completed
format.sh: 126 successful jobs, 1 failed jobs
format.sh: starting job in /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.135 (Thu May  5 13:05:46 UTC 2022)
format.sh:  catchsegv ./t -c /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/CONFIG.stress -h /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.135   quiet=1
./format.sh: line 410: wait: pid 114502 is not a child of this shell
format.sh: job in /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.132 successfully completed
format.sh: 127 successful jobs, 1 failed jobs
format.sh: starting job in /data/mci/0f76d03f2c992994263814a29d3171ca/wiredtiger/cmake_build/test/format/RUNDIR.136 (Thu May  5 13:09:40 UTC 2022)
format.sh:  catchsegv ./t -c /da
Link: https://evergreen.mongodb.com/task_log_raw/wiredtiger_ubuntu2004_stress_tests_arm64_format_stress_test_3_c5e3a8a043ba2c59c6cb3884f4d729992e2e1185_22_05_05_03_29_52/1?type=T#L1504

logs

causes

WT-10594 Disable cursor_reposition in test/format for older releases

Closed

is related to

WT-8707 Mongod reader yielding with WT::commit_transaction instead of rollback pins WT pages such that pages cannot be split/evicted by writer threads

Closed

Details

Description

Attachments

Issue Links

Activity

People

Dates