Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-957

Error on session->close

    • Type: Icon: Task Task
    • Resolution: Done
    • WT2.2.1
    • Affects Version/s: None
    • Component/s: None
    • Labels:

      I'm seeing an error on session.close:

      [1397453897:730451][12959:00b7ffe5ff7f0000], t, file:__wt0000.wt, session.close: the WiredTiger library cannot continue; the process must exit and restart
      t: session.close: WT_PANIC: WiredTiger library panic
      

      The application I'm using doesn't do anything too tricky. It starts a number of threads, each thread creates and opens a single table, and execute updates on that table. Once a certain number of updates have been completed, that thread then calls session.close. The application doesn't use explicit transactions, and also doesn't explicitly close the cursor.

      The failure reproduces quickly, but not every time I run the test. The call stack and some other information:

      (gdb) where
      #0  __wt_errx (session=session@entry=0x6b08a0, 
          fmt=fmt@entry=0x496b98 "reconciliation illegally skipped an update")
          at ../src/support/err.c:317
      WT-1  0x0000000000444c0d in __rec_write_wrapup (session=session@entry=0x6b08a0, 
          r=r@entry=0x7fffd0003cd0, page=page@entry=0x7fffd0048010)
          at ../src/btree/rec_write.c:4638
      WT-2  0x000000000044ce8f in __wt_rec_write (session=session@entry=0x6b08a0, 
          ref=ref@entry=0x7fffd0004900, salvage=salvage@entry=0x0, 
          flags=flags@entry=4) at ../src/btree/rec_write.c:413
      WT-3  0x00000000004380d4 in __evict_file (syncop=4, session=0x6b08a0)
          at ../src/btree/bt_sync.c:214
      WT-4  __wt_bt_cache_op (session=session@entry=0x6b08a0, 
          ckptbase=<optimised out>, op=op@entry=4) at ../src/btree/bt_sync.c:325
      WT-5  0x0000000000427c40 in __checkpoint_worker (session=<optimised out>, 
          cfg=cfg@entry=0x0, is_checkpoint=is_checkpoint@entry=0)
          at ../src/txn/txn_ckpt.c:761
      WT-6  0x00000000004282aa in __wt_checkpoint_close (
          session=session@entry=0x6b08a0) at ../src/txn/txn_ckpt.c:875
      WT-7  0x0000000000454e0c in __wt_conn_btree_sync_and_close (
          session=session@entry=0x6b08a0) at ../src/conn/conn_dhandle.c:208
      WT-8  0x0000000000455142 in __wt_conn_btree_close (
          session=session@entry=0x6b08a0, locked=locked@entry=0)
          at ../src/conn/conn_dhandle.c:620
      WT-9  0x000000000042271c in __wt_session_discard_btree (
          session=session@entry=0x6b08a0, dhandle_cache=0x0)
          at ../src/session/session_dhandle.c:434
      WT-10 0x0000000000421220 in __session_close_cache (session=<optimised out>)
          at ../src/session/session_api.c:39
      WT-11 __session_close (wt_session=0x6b08a0, config=<optimised out>)
          at ../src/session/session_api.c:108
      WT-12 0x0000000000402d4a in worker (arg=0x6c8ac0)
          at ../../../test/checkpoint/workers.c:202
      WT-13 0x00007ffff77b8f6e in start_thread (arg=0x7fffe5ffb700)
          at pthread_create.c:311
      WT-14 0x00007ffff74e39cd in clone ()
          at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
      (gdb) up 12
      WT-12 0x0000000000402d4a in worker (arg=0x6c8ac0)
          at ../../../test/checkpoint/workers.c:202
      202		if ((ret = session->close(session, NULL)) != 0)
      (gdb) p *cookie
      $20 = {id = 0, type = COL, uri = "table:__wt0000", '\000' <repeats 113 times>, 
        session = 0x6b08a0, cursor = 0x7fffd0004020}
      (gdb) down 12
      WT-1  0x0000000000444c0d in __rec_write_wrapup (session=session@entry=0x6b08a0, 
          r=r@entry=0x7fffd0003cd0, page=page@entry=0x7fffd0048010)
          at ../src/btree/rec_write.c:4638
      4638				WT_PANIC_RETX(session,
      (gdb) up
      WT-2  0x000000000044ce8f in __wt_rec_write (session=session@entry=0x6b08a0, 
          ref=ref@entry=0x7fffd0004900, salvage=salvage@entry=0x0, 
          flags=flags@entry=4) at ../src/btree/rec_write.c:413
      413			ret = __rec_write_wrapup(session, r, page);
      (gdb) up
      WT-3  0x00000000004380d4 in __evict_file (syncop=4, session=0x6b08a0)
          at ../src/btree/bt_sync.c:214
      214				WT_ERR(__wt_rec_write(
      (gdb) p *page
      $22 = {u = {intl = {recno = 1, parent_ref = 0x7fffd0048058, 
            __index = 0x7fffd004c9c0}, row = {d = 0x1, ins = 0x7fffd0048058, 
            upd = 0x7fffd004c9c0, entries = 2333}, col_fix = {recno = 1, 
            bitf = 0x7fffd0048058 "(", entries = 3489974720}, col_var = {recno = 1, 
            d = 0x7fffd0048058, repeats = 0x7fffd004c9c0, nrepeats = 2333, 
            entries = 4693}}, dsk = 0x7fffd0042000, modify = 0x7fffd0004270, 
        read_gen = 0, memory_footprint = 797902, type = 4 '\004', 
        flags_atomic = 18 '\022'}
      (gdb) p *page->modify
      $23 = {disk_snap_min = 610900, rec_max_txn = 0, rec_min_skipped_txn = 159807, 
        update_txn = 610869, bytes_dirty = 797902, u1 = {replace = {
            addr = 0x7fffd07a80c0 "\300\001\206\344\242Q\005i", size = 8 '\b', 
            type = 3 '\003', reuse = 0 '\000'}, m = {multi = 0x7fffd07a80c0, 
            multi_entries = 776}}, u2 = {root_split = 0x0, leaf = {append = 0x0, 
            update = 0x7fffd0058020}}, ovfl_track = 0x0, write_gen = 3228, 
        checkpoint_gen = 0, page_lock = 15 '\017', flags = 4 '\004'}
      

            Assignee:
            keith.bostic@mongodb.com Keith Bostic (Inactive)
            Reporter:
            alexander.gorrod@mongodb.com Alexander Gorrod
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: