Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-330

large numbers of overflow records slow performance

      Alex writes:

      I've been running test/format today, and have noticed that merges can take a really long time. I stopped one instance in a debugger, and saw the following call stack:

      #0  0x0000000000473026 in __wt_rec_track_ovfl_reuse (session=0x161b5e0,
          page=0x7f72283fe580, data=0x7f722809c250, data_size=1811,
          addrp=0x7f722f4c7b70, addr_sizep=0x7f722f4c7b7c, foundp=0x7f722f4c7b78)
          at ../src/btree/rec_track.c:261
      WT-1  0x00000000004424f4 in __rec_cell_build_ovfl (session=0x161b5e0,
          kv=0x7f7220000990, type=128 '\200', rle=0) at ../src/btree/rec_write.c:3849
      WT-2  0x0000000000442407 in __rec_cell_build_val (session=0x161b5e0,
          data=0x7f7215c27c00, size=1353, rle=0) at ../src/btree/rec_write.c:3810
      WT-3  0x000000000043de6b in __wt_rec_row_bulk_insert (cbulk=0x7f72241b9ce0)
          at ../src/btree/rec_write.c:1442
      WT-4  0x0000000000477446 in __wt_bulk_insert (cbulk=0x7f72241b9ce0)
          at ../src/btree/bt_bulk.c:99
      WT-5  0x0000000000474fc7 in __curbulk_insert (cursor=0x7f72241b9ce0)
          at ../src/cursor/cur_bulk.c:28
      WT-6  0x000000000045425b in __wt_lsm_major_merge (session=0x161b5e0,
          lsm_tree=0x162cdb0) at ../src/lsm/lsm_merge.c:156
      WT-7  0x0000000000412a16 in __wt_lsm_worker (arg=0x162cdb0)
          at ../src/lsm/lsm_worker.c:92

      When I looked at track_ovfl_reuse the mod->track_entries value was 295700. From what I can tell, each insert is linearly traversing the list of entries in track_ovfl_reuse only to return not found (and then grow the list).

            keith.bostic@mongodb.com Keith Bostic (Inactive)
            keith.bostic@mongodb.com Keith Bostic (Inactive)
            0 Vote for this issue
            1 Start watching this issue