Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-14092

format-stress-test-2 ran out of disk space

    • Type: Icon: Build Failure Build Failure
    • Resolution: Unresolved
    • Priority: Icon: Minor - P4 Minor - P4
    • None
    • Affects Version/s: None
    • Component/s: None
    • StorEng - Defined Pipeline

      format-stress-test-2 on ubuntu2004-release-stress-tests

      Host: i-07c8f0b9669b858f7
      Project: wiredtiger
      Commit: c50a4270

      Task Logs:

      format-stress-test-2 task_log

      Logs:

          [1727919358:196895][68744:0x7f62a14a5700], t, file:T00002.wt, WT_CURSOR.prev: [WT_VERB_DEFAULT][ERROR]: __wt_modify_reconstruct_from_upd_list, 436: Read-uncommitted readers do not support reconstructing a record with modifies.: WT_ROLLBACK: conflict between concurrent operations
          [1727919358:673236][68744:0x7f62a24a7700], t, file:T00002.wt, WT_CURSOR.prev: [WT_VERB_DEFAULT][ERROR]: __wt_modify_reconstruct_from_upd_list, 436: Read-uncommitted readers do not support reconstructing a record with modifies.: WT_ROLLBACK: conflict between concurrent operations
          [1727919359:33964][68744:0x7f62a1ca6700], t, file:T00002.wt, WT_CURSOR.prev: [WT_VERB_DEFAULT][ERROR]: __wt_modify_reconstruct_from_upd_list, 436: Read-uncommitted readers do not support reconstructing a record with modifies.: WT_ROLLBACK: conflict between concurrent operations
          [1727919398:291401][68744:0x7f62a24a7700], t, file:T00002.wt, WT_CURSOR.prev: [WT_VERB_DEFAULT][ERROR]: __wt_modify_reconstruct_from_upd_list, 436: Read-uncommitted readers do not support reconstructing a record with modifies.: WT_ROLLBACK: conflict between concurrent operations
          [1727921430:249869][68744:0x7f62a24a7700], t, file:T00002.wt, WT_CURSOR.search: [WT_VERB_DEFAULT][ERROR]: __wt_modify_reconstruct_from_upd_list, 436: Read-uncommitted readers do not support reconstructing a record with modifies.: WT_ROLLBACK: conflict between concurrent operations
      

      logs

      format-stress-test-2 task_log

      Logs:

          t: FAILED: copy_on_file/310: write(wfd, buf, (size_t)n) == n: No such file or directory
      
      

      logs

      format-stress-test-2 task_log

      Logs:

          [1727927159:757691][90032:0x7f082bfff700], t, file:T00002.wt, WT_SESSION.checkpoint: [WT_VERB_DEFAULT][ERROR]: __posix_file_write, 729: /data/mci/35e417b13957cfe0fbb23ca1eb199d10/wiredtiger/cmake_build/test/format/RUNDIR.30/T00002.wt: handle-write: pwrite: failed to write 15872 bytes at offset 274661376: No space left on device
          [1727927159:757757][90032:0x7f082bfff700], t, file:T00003.wt, WT_SESSION.checkpoint: [WT_VERB_DEFAULT][ERROR]: __wt_block_checkpoint_resolve, 1008: T00003.wt: the checkpoint failed, the system must restart: Invalid argument
          [1727927159:757765][90032:0x7f082bfff700], t, file:T00003.wt, WT_SESSION.checkpoint: [WT_VERB_DEFAULT][ERROR]: __wt_block_checkpoint_resolve, 1008: the process must exit and restart: WT_PANIC: WiredTiger library panic
          t: FAILED: [1727927159:757795][90032:0x7f0849feb700], t, connection-open-session: [WT_VERB_DEFAULT][ERROR]: __posix_file_write, 729: /data/mci/35e417b13957cfe0fbb23ca1eb199d10/wiredtiger/cmake_build/test/format/RUNDIR.30/BACKUP.copy/T00002.wt.copy: handle-write: pwrite: failed to write 131072 bytes at offset 16252928: No space left on device
      

      logs

      format-stress-test-2 task_log

      Logs:

          snap_verify_callback/267: cursor->get_value(cursor, value): WT_PANIC: WiredTiger library panic
      
      

      logs

      format-stress-test-2 task_log

      Logs:

          [1727927159:757781][90032:0x7f082bfff700], t, WT_SESSION.checkpoint: [WT_VERB_DEFAULT][ERROR]: __wt_cursor_set_key, 304: failed to set key: WT_PANIC: WiredTiger library panic
          [1727927159:757840][90032:0x7f082bfff700], t, WT_SESSION.checkpoint: [WT_VERB_DEFAULT][ERROR]: __meta_track_unroll, 222: metadata unroll update file:T00003.wt to access_pattern_hint=none,allocation_size=512,app_metadata=,assert=(commit_timestamp=none,durable_timestamp=none,read_timestamp=none,write_timestamp=off),block_allocation=best,block_compressor="lz4",cache_resident=false,checksum="uncompressed",collator=,columns=,dictionary=0,encryption=(keyid=,name=),format=btree,huffman_key=,huffman_value=,id=4,ignore_in_memory_cache_size=false,internal_item_max=0,internal_key_max=0,internal_key_truncate=true,internal_page_max=2048,key_format=r,key_gap=10,leaf_item_max=0,leaf_key_max=0,leaf_page_max=512,leaf_value_max=5120,log=(enabled=false),memory_page_image_max=0,memory_page_max=10485760,os_cache_dirty_max=0,os_cache_max=0,prefix_compression=false,prefix_compression_min=4,readonly=false,split_deepen_min_child=0,split_deepen_per_child=0,split_pct=79,tiered_object=false,tiered_storage=(auth_token=,bucket=,bucket_prefix=,cache_directory=,local_retention=300,name=,object_target_size=0),value_format=u,verbose=[],version=(major=2,minor=1),write_timestamp_usage=none,checkpoint=(WiredTigerCheckpoint.17=(addr="019181e414049920e304ab48c427e4215e4143e30801bec509e41d11a02b808080e413c2e5c0e4079e37c0",order=17,time=1727927049,size=127818240,newest_start_durable_ts=3445270,oldest_start_ts=0,newest_txn=5166084,newest_stop_durable_ts=3444741,newest_stop_ts=-1,newest_stop_txn=-11,prepare=0,write_gen=3136214,run_write_gen=19),WiredTigerCheckpoint.19=(addr="01c01181e4197da640e304b383c33ae421116585e302fea9c2abe4a607cc6de307f95dc461e4dcc1b178e413c2e5c0e407dbe3c0",order=19,time=1727927111,size=131859968,newest_start_durable_ts=3921380,oldest_start_ts=0,newest_txn=5874426,newest_stop_durable_ts=3921363,newest_stop_ts=-1,newest_stop_txn=-11,prepare=0,write_gen=3535259,run_write_gen=19)),checkpoint_backup_info=,checkpoint_lsn=(4294967295,2147483647): WT_PANIC: WiredTiger library panic
          [1727927159:757847][90032:0x7f082bfff700], t, WT_SESSION.checkpoint: [WT_VERB_DEFAULT][ERROR]: __wt_meta_track_off, 343: failed to apply or unroll all tracked operations: WT_PANIC: WiredTiger library panic
      

      logs

      format-stress-test-2 task_log

      Logs:

          t: FAILED: checkpoint/150: ret == 0 || (ret == EBUSY && ebusy_ok)
      
      

      logs

      format-stress-test-2 task_log

      Logs:

          block_cache.cache_on_checkpoint=1[1727927160:294362][90032:0x7f08592d4700], t, statlog-server: [WT_VERB_DEFAULT][ERROR]: __statlog_server, 611: statistics log server error: WT_PANIC: WiredTiger library panic
      
      

      logs

      format-stress-test-2 task_log

      Logs:

          t: process aborting
          WiredTiger Error: __wt_abort, 28: aborting WiredTiger library
      

      logs

      format-stress-test-2 task_log

      Logs:

          [1727927126:317463][90286:0x7f17ee7fc700], t, file:WiredTigerHS.wt, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __ckpt_process, 844: WiredTigerHS.wt: fatal checkpoint failure: No space left on device
          [1727927126:321109][90286:0x7f17ee7fc700], t, file:WiredTigerHS.wt, WT_SESSION.compact: [WT_VERB_DEFAULT][ERROR]: __ckpt_process, 844: the process must exit and restart: WT_PANIC: WiredTiger library panic
      

      logs

      format-stress-test-2 task_log

      Logs:

          t: FAILED: table_op/852: (1 && ret == WT_NOTFOUND) || ret == WT_CACHE_FULL || ret == WT_ROLLBACK: operation failed: -31804
      
      

      logs

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Xgen-BuildBaron-User xgen-buildbaron-user
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: