Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-894

test/format eviction hang

    • Type: Icon: Task Task
    • Resolution: Done
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:

      Since I was able to reproduce WT-884 on the AWS HD system, and there was a merge of a fix, I started it off yesterday to confirm the fix. It hung in eviction. I'm running with changeset 0a9f55477e7bd195f455d2b1cba834b68cbab4f6.

      Here is the stack for the eviction thread:

      Thread 28 (Thread 0x7f84fbf26700 (LWP 25733)):
      #0  0x00007f84fd27fe2e in pthread_cond_timedwait@@GLIBC_2.3.2 ()
         from /lib64/libpthread.so.0
      WT-1  0x0000000000422577 in __wt_cond_wait (session=0xa95550, cond=0xaadf30, 
          usecs=100000) at ../src/os_posix/os_mtx.c:89
      WT-2  0x0000000000445317 in __wt_cache_evict_server (arg=0xa95550)
          at ../src/btree/bt_evict.c:172
      WT-3  0x00007f84fd27bddb in start_thread () from /lib64/libpthread.so.0
      

      Every other thread is in __wt_cache_full_check waiting on the condition. The top of every other stack looks like:

      #0  0x00007f84fd27fe2e in pthread_cond_timedwait@@GLIBC_2.3.2 ()
         from /lib64/libpthread.so.0
      WT-1  0x0000000000422577 in __wt_cond_wait (session=0xa957d0, cond=0xaadfa0, 
          usecs=100000) at ../src/os_posix/os_mtx.c:89
      WT-2  0x00000000004a0219 in __wt_cache_full_check (session=0xa957d0)
          at ../src/include/cache.i:124
      

      Here is the state of the cache. Flags are NO_PROGRESS | STUCK.

      (gdb) p/x *cache
      $2 = {bytes_inmem = 0x37998cd49, pages_inmem = 0x59c79, 
        bytes_evict = 0x379363da8, pages_evict = 0x59bc8, bytes_dirty = 0x0, 
        pages_dirty = 0x0, read_gen = 0xc03a6, evict_cond = 0xaadf30, evict_lock = {
          lock = {__data = {__lock = 0x0, __count = 0x0, __owner = 0x0, 
              __nusers = 0x0, __kind = 0x3, __spins = 0x8, __list = {__prev = 0x0, 
                __next = 0x0}}, __size = {0x0 <repeats 16 times>, 0x3, 0x0, 0x0, 
              0x0, 0x8, 0x0 <repeats 19 times>}, __align = 0x0}, counter = 0x0, 
          name = 0x4ba0bf, id = 0x0, initialized = 0x1}, evict_walk_lock = {lock = {
            __data = {__lock = 0x0, __count = 0x0, __owner = 0x0, __nusers = 0x0, 
              __kind = 0x3, __spins = 0x58, __list = {__prev = 0x0, __next = 0x0}}, 
            __size = {0x0 <repeats 16 times>, 0x3, 0x0, 0x0, 0x0, 0x58, 
              0x0 <repeats 19 times>}, __align = 0x0}, counter = 0x0, 
          name = 0x4ba0ce, id = 0x0, initialized = 0x1}, 
        evict_waiter_cond = 0xaadfa0, eviction_trigger = 0x5f, 
        eviction_target = 0x50, eviction_dirty_target = 0x50, evict = 0xaaf420, 
        evict_current = 0x0, evict_candidates = 0xe, evict_entries = 0x27, 
        evict_max = 0x138, evict_slots = 0x190, evict_file_next = 0x0, 
        sync_request = 0x0, sync_complete = 0x0, cp_saved_evict = 0x0, 
        cp_current_evict = 0x0, cp_skip_count = 0x0, cp_reserved = 0xa00000, 
        flags = 0xc}
      

            Assignee:
            Unassigned Unassigned
            Reporter:
            sue.loverso@mongodb.com Susan LoVerso
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: