Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-5069

test/format hung on PPC in Jenkins

    • Type: Icon: Bug Bug
    • Resolution: Gone away
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:

      The Jenkins job http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-ppc/21840/ has been running and hung for 32 hours. I am going to kill it but here is the stack. Eviction is stuck but test/format is not triggering its 15 minute timeout because that is wts_ops and we haven't gotten to that yet.

      Thread 4 (Thread 0x3fff9992f1b0 (LWP 39121)):
      #0  0x00003fff99f2dfb4 in pthread_cond_timedwait@@GLIBC_2.17 ()
         from /lib64/power8/libpthread.so.0
      #1  0x00000000100981a0 in __wt_cond_wait_signal (session=0x3fff99992590, 
          cond=0x10022cbc1f0, usecs=1000000, run_func=0x0, signalled=0x3fff9992e650)
          at ../src/os_posix/os_mtx_cond.c:122
      #2  0x00000000100e1bb0 in __wt_cond_auto_wait_signal (session=0x3fff99992590, 
          cond=0x10022cbc1f0, progress=false, run_func=0x0, signalled=0x3fff9992e650)
          at ../src/support/cond_auto.c:62
      #3  0x00000000100e1ce4 in __wt_cond_auto_wait (session=0x3fff99992590, 
          cond=0x10022cbc1f0, progress=false, run_func=0x0)
          at ../src/support/cond_auto.c:82
      #4  0x000000001006761c in __wt_evict_thread_run (session=0x3fff99992590, 
          thread=0x10022d05db0) at ../src/evict/evict_lru.c:331
      #5  0x00000000100f8578 in __thread_run (arg=0x10022d05db0)
          at ../src/support/thread_group.c:31
      #6  0x00003fff99f28944 in start_thread () from /lib64/power8/libpthread.so.0
      #7  0x00003fff99da7640 in clone () from /lib64/power8/libc.so.6
      
      Thread 3 (Thread 0x3fff9912f1b0 (LWP 39122)):
      #0  0x00003fff99f2dfb4 in pthread_cond_timedwait@@GLIBC_2.17 ()
         from /lib64/power8/libpthread.so.0
      #1  0x00000000100981a0 in __wt_cond_wait_signal (session=0x3fff99992a40, 
          cond=0x10022ce8240, usecs=2000000, 
          run_func=0x10051424 <__sweep_server_run_chk>, signalled=0x3fff9912e7b4)
          at ../src/os_posix/os_mtx_cond.c:122
      #2  0x00000000100515b8 in __sweep_server (arg=0x3fff99992a40)
          at ../src/conn/conn_sweep.c:306
      #3  0x00003fff99f28944 in start_thread () from /lib64/power8/libpthread.so.0
      #4  0x00003fff99da7640 in clone () from /lib64/power8/libc.so.6
      
      Thread 2 (Thread 0x3fff9892f1b0 (LWP 39123)):
      #0  0x00003fff99f2dfb4 in pthread_cond_timedwait@@GLIBC_2.17 ()
         from /lib64/power8/libpthread.so.0
      #1  0x00000000100981a0 in __wt_cond_wait_signal (session=0x3fff99992ef0, 
          cond=0x10022cf27e0, usecs=33000000, 
          run_func=0x101bafdc <__ckpt_server_run_chk>, signalled=0x3fff9892e6f0)
          at ../src/os_posix/os_mtx_cond.c:122
      #2  0x00000000101bad78 in __wt_cond_wait (session=0x3fff99992ef0, 
          cond=0x10022cf27e0, usecs=33000000, 
          run_func=0x101bafdc <__ckpt_server_run_chk>) at ../src/include/misc.i:19
      #3  0x00000000101bb0a0 in __ckpt_server (arg=0x3fff99992ef0)
          at ../src/conn/conn_ckpt.c:98
      #4  0x00003fff99f28944 in start_thread () from /lib64/power8/libpthread.so.0
      #5  0x00003fff99da7640 in clone () from /lib64/power8/libc.so.6
      
      Thread 1 (Thread 0x3fff9a067070 (LWP 39110)):
      #0  0x00003fff99f2dfb4 in pthread_cond_timedwait@@GLIBC_2.17 ()
         from /lib64/power8/libpthread.so.0
      #1  0x00000000100981a0 in __wt_cond_wait_signal (session=0x3fff999933a0, 
          cond=0x10022cf50e0, usecs=10000, run_func=0x0, signalled=0x3fffcf5a9810)
          at ../src/os_posix/os_mtx_cond.c:122
      #2  0x0000000010064460 in __wt_cond_wait (session=0x3fff999933a0, 
          cond=0x10022cf50e0, usecs=10000, run_func=0x0) at ../src/include/misc.i:19
      #3  0x000000001006e234 in __wt_cache_eviction_worker (session=0x3fff999933a0, 
          busy=false, readonly=false, pct_full=154.78749851807956)
          at ../src/evict/evict_lru.c:2518
      #4  0x00000000101404bc in __wt_cache_eviction_check (session=0x3fff999933a0, 
          busy=false, readonly=false, didworkp=0x0) at ../src/include/cache.i:453
      #5  0x0000000010143218 in __cursor_enter (session=0x3fff999933a0)
          at ../src/include/cursor.i:152
      #6  0x000000001014368c in __cursor_func_init (cbt=0x10024f5f320, reenter=true)
          at ../src/include/cursor.i:359
      #7  0x0000000010144d18 in __wt_btcur_search (cbt=0x10024f5f320)
          at ../src/btree/bt_cursor.c:591
      #8  0x00000000101d39ec in __curfile_search (cursor=0x10024f5f320)
          at ../src/cursor/cur_file.c:200
      #9  0x00000000100097a0 in read_op (cursor=0x10024f5f320, op=SEARCH, exactp=0x0)
          at ../../../test/format/format.i:53
      #10 0x000000001000dd00 in read_row_worker (cursor=0x10024f5f320, keyno=51540, 
          key=0x3fffcf5a9da8, value=0x3fffcf5a9dd0, sn=false)
          at ../../../test/format/ops.c:1146
      #11 0x000000001000daa4 in wts_read_scan () at ../../../test/format/ops.c:1096
      #12 0x000000001001462c in main (argc=4, argv=0x3fffcf5aa328)
          at ../../../test/format/t.c:218
      

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            sue.loverso@mongodb.com Susan LoVerso
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: