Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-1613

test/format stress hang - eviction stuck

    • Type: Icon: Task Task
    • Resolution: Done
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:

      Eviction is stuck and this was hung for 18 hours. I collected basic info here and then killed the job so that we could run test/format with the new cache accounting changes today. Cache here is tiny, 2Mb.

      This could be a duplicate of WT-1522 given the tiny cache.

      ############################################
      #  RUN PARAMETERS
      ############################################
      abort=0
      auto_throttle=1
      firstfit=0
      bitcnt=3
      bloom=1
      bloom_bit_count=58
      bloom_hash_count=31
      bloom_oldest=0
      cache=2
      checkpoints=1
      checksum=uncompressed
      chunk_size=2
      compaction=0
      compression=zlib
      data_extend=0
      data_source=file
      delete_pct=42
      dictionary=0
      evict_max=2
      file_type=row-store
      backups=0
      huffman_key=0
      huffman_value=0
      insert_pct=18
      internal_key_truncation=1
      internal_page_max=17
      isolation=snapshot
      key_gap=4
      key_max=125
      key_min=10
      leak_memory=0
      leaf_page_max=11
      logging=1
      logging_archive=0
      logging_prealloc=0
      logging=1
      lsm_worker_threads=3
      merge_max=12
      mmap=1
      ops=100000
      prefix_compression=1
      prefix_compression_min=6
      repeat_data_pct=26
      reverse=0
      rows=100000
      runs=100
      split_pct=46
      statistics=1
      threads=12
      timer=0
      value_max=3171
      value_min=0
      wiredtiger_config=
      write_pct=68
      ############################################
      
      (gdb) thread apply all bt
      
      Thread 16 (Thread 0x7f959ecbe700 (LWP 24869)):
      #0  0x0000003467a0b98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      WT-1  0x00000000004410ec in __wt_cond_wait (session=0x16be150, cond=0x16bb690, usecs=100000) at ../src/os_posix/os_mtx_cond.c:78
      WT-2  0x000000000042a4b9 in __evict_server (arg=0x16be150) at ../src/evict/evict_lru.c:196
      WT-3  0x0000003467a079d1 in start_thread () from /lib64/libpthread.so.0
      WT-4  0x00000034676e8b6d in clone () from /lib64/libc.so.6
      
      Thread 15 (Thread 0x7f959d8bc700 (LWP 24870)):
      #0  0x0000003467a0b98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      WT-1  0x00000000004410ec in __wt_cond_wait (session=0x16be6d0, cond=0x16f8230, usecs=10000000) at ../src/os_posix/os_mtx_cond.c:78
      WT-2  0x0000000000420cc6 in __sweep_server (arg=0x16be6d0) at ../src/conn/conn_sweep.c:123
      WT-3  0x0000003467a079d1 in start_thread () from /lib64/libpthread.so.0
      WT-4  0x00000034676e8b6d in clone () from /lib64/libc.so.6
      
      Thread 14 (Thread 0x7f955e1fc700 (LWP 24872)):
      #0  0x0000003467a0b98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      WT-1  0x00000000004410ec in __wt_cond_wait (session=0x16be990, cond=0x16f9030, usecs=10000) at ../src/os_posix/os_mtx_cond.c:78
      WT-2  0x00000000004a9ae3 in __log_close_server (arg=0x16be990) at ../src/conn/conn_log.c:334
      WT-3  0x0000003467a079d1 in start_thread () from /lib64/libpthread.so.0
      WT-4  0x00000034676e8b6d in clone () from /lib64/libc.so.6
      
      Thread 13 (Thread 0x7f959cebb700 (LWP 30804)):
      #0  0x0000003467a0b98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      WT-1  0x00000000004410ec in __wt_cond_wait (session=0x16bff90, cond=0x16bb700, usecs=100000) at ../src/os_posix/os_mtx_cond.c:78
      WT-2  0x000000000042cbd0 in __wt_cache_wait (session=0x16bff90, full=103) at ../src/evict/evict_lru.c:1399
      WT-3  0x00000000004e47c3 in __wt_cache_full_check (session=0x16bff90) at ../src/include/cache.i:176
      WT-4  0x00000000004e50c0 in __cursor_enter (session=0x16bff90) at ../src/include/cursor.i:63
      WT-5  0x00000000004e518d in __curfile_enter (cbt=0x7f956c075a90) at ../src/include/cursor.i:98
      WT-6  0x00000000004e5357 in __cursor_func_init (cbt=0x7f956c075a90, reenter=1) at ../src/include/cursor.i:197
      WT-7  0x00000000004e5ad7 in __wt_btcur_search (cbt=0x7f956c075a90) at ../src/btree/bt_cursor.c:308
      WT-8  0x00000000004b20d0 in __curfile_search (cursor=0x7f956c075a90) at ../src/cursor/cur_file.c:200
      WT-9  0x0000000000410a21 in read_row (cursor=0x7f956c075a90, key=0x7f959cebae10, keyno=49148) at ../../../test/format/ops.c:602
      WT-10 0x000000000041055f in ops (arg=0x196b720) at ../../../test/format/ops.c:476
      WT-11 0x0000003467a079d1 in start_thread () from /lib64/libpthread.so.0
      WT-12 0x00000034676e8b6d in clone () from /lib64/libc.so.6
      
      Thread 12 (Thread 0x7f9597fff700 (LWP 30805)):
      #0  0x0000003467a0b98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      WT-1  0x00000000004410ec in __wt_cond_wait (session=0x16bec50, cond=0x16bb700, usecs=100000) at ../src/os_posix/os_mtx_cond.c:78
      WT-2  0x000000000042cbd0 in __wt_cache_wait (session=0x16bec50, full=103) at ../src/evict/evict_lru.c:1399
      WT-3  0x00000000004e47c3 in __wt_cache_full_check (session=0x16bec50) at ../src/include/cache.i:176
      WT-4  0x00000000004e50c0 in __cursor_enter (session=0x16bec50) at ../src/include/cursor.i:63
      WT-5  0x00000000004e518d in __curfile_enter (cbt=0x7f954c06d4d0) at ../src/include/cursor.i:98
      WT-6  0x00000000004e5357 in __cursor_func_init (cbt=0x7f954c06d4d0, reenter=1) at ../src/include/cursor.i:197
      WT-7  0x00000000004e5ad7 in __wt_btcur_search (cbt=0x7f954c06d4d0) at ../src/btree/bt_cursor.c:308
      WT-8  0x00000000004b20d0 in __curfile_search (cursor=0x7f954c06d4d0) at ../src/cursor/cur_file.c:200
      WT-9  0x0000000000411cc7 in row_remove (cursor=0x7f954c06d4d0, key=0x7f9597ffee10, keyno=13118, notfoundp=0x7f9597ffedc4) at ../../../test/format/ops.c:1056
      WT-10 0x00000000004102a2 in ops (arg=0x196b778) at ../../../test/format/ops.c:399
      WT-11 0x0000003467a079d1 in start_thread () from /lib64/libpthread.so.0
      WT-12 0x00000034676e8b6d in clone () from /lib64/libc.so.6
      
      Thread 11 (Thread 0x7f9596bfd700 (LWP 30806)):
      #0  0x0000003467a0b98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      WT-1  0x00000000004410ec in __wt_cond_wait (session=0x16bef10, cond=0x16bb700, usecs=100000) at ../src/os_posix/os_mtx_cond.c:78
      WT-2  0x000000000042cbd0 in __wt_cache_wait (session=0x16bef10, full=103) at ../src/evict/evict_lru.c:1399
      WT-3  0x00000000004e47c3 in __wt_cache_full_check (session=0x16bef10) at ../src/include/cache.i:176
      WT-4  0x00000000004e50c0 in __cursor_enter (session=0x16bef10) at ../src/include/cursor.i:63
      WT-5  0x00000000004e518d in __curfile_enter (cbt=0x7f95380fe090) at ../src/include/cursor.i:98
      WT-6  0x00000000004e5357 in __cursor_func_init (cbt=0x7f95380fe090, reenter=1) at ../src/include/cursor.i:197
      WT-7  0x00000000004e6aa0 in __wt_btcur_update (cbt=0x7f95380fe090) at ../src/btree/bt_cursor.c:700
      WT-8  0x00000000004b3326 in __curfile_update (cursor=0x7f95380fe090) at ../src/cursor/cur_file.c:284
      WT-9  0x000000000041131a in row_update (cursor=0x7f95380fe090, key=0x7f9596bfce10, value=0x7f9596bfcde0, keyno=78469) at ../../../test/format/ops.c:778
      WT-10 0x000000000041044e in ops (arg=0x196b7d0) at ../../../test/format/ops.c:442
      WT-11 0x0000003467a079d1 in start_thread () from /lib64/libpthread.so.0
      WT-12 0x00000034676e8b6d in clone () from /lib64/libc.so.6
      
      Thread 10 (Thread 0x7f959e2bd700 (LWP 30807)):
      #0  0x0000003467a0b98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      WT-1  0x00000000004410ec in __wt_cond_wait (session=0x16bf1d0, cond=0x16bb700, usecs=100000) at ../src/os_posix/os_mtx_cond.c:78
      WT-2  0x000000000042cbd0 in __wt_cache_wait (session=0x16bf1d0, full=103) at ../src/evict/evict_lru.c:1399
      WT-3  0x00000000004e47c3 in __wt_cache_full_check (session=0x16bf1d0) at ../src/include/cache.i:176
      WT-4  0x00000000004e50c0 in __cursor_enter (session=0x16bf1d0) at ../src/include/cursor.i:63
      WT-5  0x00000000004e518d in __curfile_enter (cbt=0x7f9574044010) at ../src/include/cursor.i:98
      WT-6  0x00000000004e5357 in __cursor_func_init (cbt=0x7f9574044010, reenter=1) at ../src/include/cursor.i:197
      WT-7  0x00000000004e5ad7 in __wt_btcur_search (cbt=0x7f9574044010) at ../src/btree/bt_cursor.c:308
      WT-8  0x00000000004b20d0 in __curfile_search (cursor=0x7f9574044010) at ../src/cursor/cur_file.c:200
      WT-9  0x0000000000410a21 in read_row (cursor=0x7f9574044010, key=0x7f959e2bce10, keyno=26422) at ../../../test/format/ops.c:602
      WT-10 0x000000000041055f in ops (arg=0x196b828) at ../../../test/format/ops.c:476
      WT-11 0x0000003467a079d1 in start_thread () from /lib64/libpthread.so.0
      WT-12 0x00000034676e8b6d in clone () from /lib64/libc.so.6
      
      Thread 9 (Thread 0x7f95975fe700 (LWP 30808)):
      #0  0x0000003467a0b98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      WT-1  0x00000000004410ec in __wt_cond_wait (session=0x16c0510, cond=0x16bb700, usecs=100000) at ../src/os_posix/os_mtx_cond.c:78
      WT-2  0x000000000042cbd0 in __wt_cache_wait (session=0x16c0510, full=103) at ../src/evict/evict_lru.c:1399
      WT-3  0x00000000004e47c3 in __wt_cache_full_check (session=0x16c0510) at ../src/include/cache.i:176
      WT-4  0x00000000004e50c0 in __cursor_enter (session=0x16c0510) at ../src/include/cursor.i:63
      WT-5  0x00000000004e518d in __curfile_enter (cbt=0x7f959803aad0) at ../src/include/cursor.i:98
      WT-6  0x00000000004e5357 in __cursor_func_init (cbt=0x7f959803aad0, reenter=1) at ../src/include/cursor.i:197
      WT-7  0x00000000004e6aa0 in __wt_btcur_update (cbt=0x7f959803aad0) at ../src/btree/bt_cursor.c:700
      WT-8  0x00000000004b3326 in __curfile_update (cursor=0x7f959803aad0) at ../src/cursor/cur_file.c:284
      WT-9  0x000000000041131a in row_update (cursor=0x7f959803aad0, key=0x7f95975fde10, value=0x7f95975fdde0, keyno=49103) at ../../../test/format/ops.c:778
      WT-10 0x000000000041044e in ops (arg=0x196b880) at ../../../test/format/ops.c:442
      WT-11 0x0000003467a079d1 in start_thread () from /lib64/libpthread.so.0
      WT-12 0x00000034676e8b6d in clone () from /lib64/libc.so.6
      
      Thread 8 (Thread 0x7f95961fc700 (LWP 30809)):
      #0  0x0000003467a0b98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      WT-1  0x00000000004410ec in __wt_cond_wait (session=0x16bf490, cond=0x16bb700, usecs=100000) at ../src/os_posix/os_mtx_cond.c:78
      WT-2  0x000000000042cbd0 in __wt_cache_wait (session=0x16bf490, full=103) at ../src/evict/evict_lru.c:1399
      WT-3  0x00000000004e47c3 in __wt_cache_full_check (session=0x16bf490) at ../src/include/cache.i:176
      WT-4  0x00000000004e50c0 in __cursor_enter (session=0x16bf490) at ../src/include/cursor.i:63
      WT-5  0x00000000004e518d in __curfile_enter (cbt=0x7f958403e9f0) at ../src/include/cursor.i:98
      WT-6  0x00000000004e5357 in __cursor_func_init (cbt=0x7f958403e9f0, reenter=1) at ../src/include/cursor.i:197
      WT-7  0x00000000004e611b in __wt_btcur_insert (cbt=0x7f958403e9f0) at ../src/btree/bt_cursor.c:448
      WT-8  0x00000000004b2b82 in __curfile_insert (cursor=0x7f958403e9f0) at ../src/cursor/cur_file.c:245
      WT-9  0x0000000000411923 in row_insert (cursor=0x7f958403e9f0, key=0x7f95961fbe10, value=0x7f95961fbde0, keyno=7476) at ../../../test/format/ops.c:970
      WT-10 0x000000000041034a in ops (arg=0x196b8d8) at ../../../test/format/ops.c:412
      WT-11 0x0000003467a079d1 in start_thread () from /lib64/libpthread.so.0
      WT-12 0x00000034676e8b6d in clone () from /lib64/libc.so.6
      
      Thread 7 (Thread 0x7f95957fb700 (LWP 30810)):
      #0  0x0000003467a0b98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      WT-1  0x00000000004410ec in __wt_cond_wait (session=0x16bf750, cond=0x16bb700, usecs=100000) at ../src/os_posix/os_mtx_cond.c:78
      WT-2  0x000000000042cbd0 in __wt_cache_wait (session=0x16bf750, full=103) at ../src/evict/evict_lru.c:1399
      WT-3  0x00000000004e47c3 in __wt_cache_full_check (session=0x16bf750) at ../src/include/cache.i:176
      WT-4  0x00000000004e50c0 in __cursor_enter (session=0x16bf750) at ../src/include/cursor.i:63
      WT-5  0x00000000004e518d in __curfile_enter (cbt=0x7f950801b350) at ../src/include/cursor.i:98
      WT-6  0x00000000004e5357 in __cursor_func_init (cbt=0x7f950801b350, reenter=1) at ../src/include/cursor.i:197
      WT-7  0x00000000004e5ad7 in __wt_btcur_search (cbt=0x7f950801b350) at ../src/btree/bt_cursor.c:308
      WT-8  0x00000000004b20d0 in __curfile_search (cursor=0x7f950801b350) at ../src/cursor/cur_file.c:200
      WT-9  0x0000000000410a21 in read_row (cursor=0x7f950801b350, key=0x7f95957fae10, keyno=31699) at ../../../test/format/ops.c:602
      WT-10 0x000000000041055f in ops (arg=0x196b930) at ../../../test/format/ops.c:476
      WT-11 0x0000003467a079d1 in start_thread () from /lib64/libpthread.so.0
      WT-12 0x00000034676e8b6d in clone () from /lib64/libc.so.6
      
      Thread 6 (Thread 0x7f9594dfa700 (LWP 30811)):
      #0  0x0000003467a0b98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      WT-1  0x00000000004410ec in __wt_cond_wait (session=0x16c07d0, cond=0x16bb700, usecs=100000) at ../src/os_posix/os_mtx_cond.c:78
      WT-2  0x000000000042cbd0 in __wt_cache_wait (session=0x16c07d0, full=103) at ../src/evict/evict_lru.c:1399
      WT-3  0x00000000004e47c3 in __wt_cache_full_check (session=0x16c07d0) at ../src/include/cache.i:176
      WT-4  0x00000000004e50c0 in __cursor_enter (session=0x16c07d0) at ../src/include/cursor.i:63
      WT-5  0x00000000004e518d in __curfile_enter (cbt=0x7f958c05d300) at ../src/include/cursor.i:98
      WT-6  0x00000000004e5357 in __cursor_func_init (cbt=0x7f958c05d300, reenter=1) at ../src/include/cursor.i:197
      WT-7  0x00000000004e611b in __wt_btcur_insert (cbt=0x7f958c05d300) at ../src/btree/bt_cursor.c:448
      WT-8  0x00000000004b2b82 in __curfile_insert (cursor=0x7f958c05d300) at ../src/cursor/cur_file.c:245
      WT-9  0x0000000000411923 in row_insert (cursor=0x7f958c05d300, key=0x7f9594df9e10, value=0x7f9594df9de0, keyno=9775) at ../../../test/format/ops.c:970
      WT-10 0x000000000041034a in ops (arg=0x196b988) at ../../../test/format/ops.c:412
      WT-11 0x0000003467a079d1 in start_thread () from /lib64/libpthread.so.0
      WT-12 0x00000034676e8b6d in clone () from /lib64/libc.so.6
      
      Thread 5 (Thread 0x7f957ffff700 (LWP 30812)):
      #0  0x0000003467a0b98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      WT-1  0x00000000004410ec in __wt_cond_wait (session=0x16c0250, cond=0x16bb700, usecs=100000) at ../src/os_posix/os_mtx_cond.c:78
      WT-2  0x000000000042cbd0 in __wt_cache_wait (session=0x16c0250, full=103) at ../src/evict/evict_lru.c:1399
      WT-3  0x00000000004e47c3 in __wt_cache_full_check (session=0x16c0250) at ../src/include/cache.i:176
      WT-4  0x00000000004e50c0 in __cursor_enter (session=0x16c0250) at ../src/include/cursor.i:63
      WT-5  0x00000000004e518d in __curfile_enter (cbt=0x7f951803b270) at ../src/include/cursor.i:98
      WT-6  0x00000000004e5357 in __cursor_func_init (cbt=0x7f951803b270, reenter=1) at ../src/include/cursor.i:197
      WT-7  0x00000000004e5ad7 in __wt_btcur_search (cbt=0x7f951803b270) at ../src/btree/bt_cursor.c:308
      WT-8  0x00000000004b20d0 in __curfile_search (cursor=0x7f951803b270) at ../src/cursor/cur_file.c:200
      WT-9  0x0000000000410a21 in read_row (cursor=0x7f951803b270, key=0x7f957fffee10, keyno=57082) at ../../../test/format/ops.c:602
      WT-10 0x000000000041055f in ops (arg=0x196b9e0) at ../../../test/format/ops.c:476
      WT-11 0x0000003467a079d1 in start_thread () from /lib64/libpthread.so.0
      WT-12 0x00000034676e8b6d in clone () from /lib64/libc.so.6
      
      Thread 4 (Thread 0x7f957f5fe700 (LWP 30813)):
      #0  0x0000003467a0b98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      WT-1  0x00000000004410ec in __wt_cond_wait (session=0x16c0a90, cond=0x16bb700, usecs=100000) at ../src/os_posix/os_mtx_cond.c:78
      WT-2  0x000000000042cbd0 in __wt_cache_wait (session=0x16c0a90, full=103) at ../src/evict/evict_lru.c:1399
      WT-3  0x000000000048637d in __wt_cache_full_check (session=0x16c0a90) at ../src/include/cache.i:176
      WT-4  0x00000000004873ea in __wt_page_in_func (session=0x16c0a90, ref=0x7f956803d190, flags=0, file=0x694fa0 "../src/btree/row_srch.c", line=296)
          at ../src/btree/bt_page.c:88
      WT-5  0x00000000004a221f in __wt_page_swap_func (session=0x16c0a90, held=0x7f95900a9b80, want=0x7f956803d190, flags=0, file=0x694fa0 "../src/btree/row_srch.c", 
          line=296) at ../src/include/btree.i:986
      WT-6  0x00000000004a2e8c in __wt_row_search (session=0x16c0a90, srch_key=0x7f956809ddb8, leaf=0x0, cbt=0x7f956809dce0, insert=0) at ../src/btree/row_srch.c:296
      WT-7  0x00000000004e5872 in __cursor_row_search (session=0x16c0a90, cbt=0x7f956809dce0, insert=0) at ../src/btree/bt_cursor.c:239
      WT-8  0x00000000004e67ff in __wt_btcur_remove (cbt=0x7f956809dce0) at ../src/btree/bt_cursor.c:638
      WT-9  0x00000000004b3ede in __curfile_remove (cursor=0x7f956809dce0) at ../src/cursor/cur_file.c:331
      WT-10 0x0000000000411ce4 in row_remove (cursor=0x7f956809dce0, key=0x7f957f5fde10, keyno=46779, notfoundp=0x7f957f5fddc4) at ../../../test/format/ops.c:1057
      WT-11 0x00000000004102a2 in ops (arg=0x196ba38) at ../../../test/format/ops.c:399
      WT-12 0x0000003467a079d1 in start_thread () from /lib64/libpthread.so.0
      WT-13 0x00000034676e8b6d in clone () from /lib64/libc.so.6
      
      Thread 3 (Thread 0x7f957ebfd700 (LWP 30814)):
      #0  0x0000003467a0b98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      WT-1  0x00000000004410ec in __wt_cond_wait (session=0x16bfcd0, cond=0x16bb700, usecs=100000) at ../src/os_posix/os_mtx_cond.c:78
      WT-2  0x000000000042cbd0 in __wt_cache_wait (session=0x16bfcd0, full=103) at ../src/evict/evict_lru.c:1399
      WT-3  0x00000000004e47c3 in __wt_cache_full_check (session=0x16bfcd0) at ../src/include/cache.i:176
      WT-4  0x00000000004e50c0 in __cursor_enter (session=0x16bfcd0) at ../src/include/cursor.i:63
      WT-5  0x00000000004e518d in __curfile_enter (cbt=0x7f951003b810) at ../src/include/cursor.i:98
      WT-6  0x00000000004e5357 in __cursor_func_init (cbt=0x7f951003b810, reenter=1) at ../src/include/cursor.i:197
      WT-7  0x00000000004e6aa0 in __wt_btcur_update (cbt=0x7f951003b810) at ../src/btree/bt_cursor.c:700
      WT-8  0x00000000004b3326 in __curfile_update (cursor=0x7f951003b810) at ../src/cursor/cur_file.c:284
      WT-9  0x000000000041131a in row_update (cursor=0x7f951003b810, key=0x7f957ebfce10, value=0x7f957ebfcde0, keyno=83007) at ../../../test/format/ops.c:778
      WT-10 0x000000000041044e in ops (arg=0x196ba90) at ../../../test/format/ops.c:442
      WT-11 0x0000003467a079d1 in start_thread () from /lib64/libpthread.so.0
      WT-12 0x00000034676e8b6d in clone () from /lib64/libc.so.6
      
      Thread 2 (Thread 0x7f957e1fc700 (LWP 30815)):
      #0  0x0000003467a0b98e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      WT-1  0x00000000004410ec in __wt_cond_wait (session=0x16bfa10, cond=0x16bb700, usecs=100000) at ../src/os_posix/os_mtx_cond.c:78
      WT-2  0x000000000042cbd0 in __wt_cache_wait (session=0x16bfa10, full=103) at ../src/evict/evict_lru.c:1399
      WT-3  0x00000000004e47c3 in __wt_cache_full_check (session=0x16bfa10) at ../src/include/cache.i:176
      WT-4  0x00000000004e50c0 in __cursor_enter (session=0x16bfa10) at ../src/include/cursor.i:63
      WT-5  0x00000000004e518d in __curfile_enter (cbt=0x7f9504066a70) at ../src/include/cursor.i:98
      WT-6  0x00000000004e5357 in __cursor_func_init (cbt=0x7f9504066a70, reenter=1) at ../src/include/cursor.i:197
      WT-7  0x00000000004e5ad7 in __wt_btcur_search (cbt=0x7f9504066a70) at ../src/btree/bt_cursor.c:308
      WT-8  0x00000000004b20d0 in __curfile_search (cursor=0x7f9504066a70) at ../src/cursor/cur_file.c:200
      WT-9  0x0000000000410a21 in read_row (cursor=0x7f9504066a70, key=0x7f957e1fbe10, keyno=14522) at ../../../test/format/ops.c:602
      WT-10 0x000000000041055f in ops (arg=0x196bae8) at ../../../test/format/ops.c:476
      WT-11 0x0000003467a079d1 in start_thread () from /lib64/libpthread.so.0
      WT-12 0x00000034676e8b6d in clone () from /lib64/libc.so.6
      ---Type <return> to continue, or q <return> to quit---
      
      Thread 1 (Thread 0x7f959f0ce700 (LWP 8799)):
      #0  0x00000034676accdd in nanosleep () from /lib64/libc.so.6
      WT-1  0x00000034676e1e54 in usleep () from /lib64/libc.so.6
      WT-2  0x000000000040fa96 in wts_ops () at ../../../test/format/ops.c:162
      WT-3  0x0000000000412aed in main (argc=0, argv=0x7fff4962c0e0) at ../../../test/format/t.c:192
      (gdb) p/x *cache
      $2 = {bytes_inmem = 0x46b5712e5, pages_inmem = 0x591cc, bytes_evict = 0x46b389735, pages_evict = 0x591c8, bytes_dirty = 0x1e7ab7, pages_dirty = 0x3, 
        evict_max_page_size = 0x19e777, read_gen = 0xb0cbc, evict_cond = 0x16bb690, evict_lock = {lock = {__data = {__lock = 0x0, __count = 0x0, __owner = 0x0, 
              __nusers = 0x0, __kind = 0x0, __spins = 0x0, __list = {__prev = 0x0, __next = 0x0}}, __size = {0x0 <repeats 40 times>}, __align = 0x0}, counter = 0x0, 
          name = 0x68490f, id = 0x0, initialized = 0x1}, evict_walk_lock = {lock = {__data = {__lock = 0x0, __count = 0x0, __owner = 0x0, __nusers = 0x0, 
              __kind = 0x0, __spins = 0x0, __list = {__prev = 0x0, __next = 0x0}}, __size = {0x0 <repeats 40 times>}, __align = 0x0}, counter = 0x0, name = 0x68491e, 
          id = 0x0, initialized = 0x1}, evict_waiter_cond = 0x16bb700, eviction_trigger = 0x5f, eviction_target = 0x50, eviction_dirty_target = 0x50, 
        evict = 0x16d8e90, evict_current = 0x0, evict_candidates = 0x1, evict_entries = 0x1, evict_max = 0x190, evict_slots = 0x190, evict_file_next = 0x0, 
        sync_request = 0x0, sync_complete = 0x0, cp_saved_evict = 0x0, cp_current_evict = 0x0, cp_skip_count = 0x0, cp_reserved = 0x0, cp_session = 0x0, cp_tid = 0x0, 
        flags = 0x10}
      

            Assignee:
            Unassigned Unassigned
            Reporter:
            sue.loverso@mongodb.com Susan LoVerso
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved: