Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-1296

Heap use after free

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Closed
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      There was a Jenkins failure with address sanitizer. Details:

      nice ./t
      t: process 18828
      =================================================================
      ==18828==ERROR: AddressSanitizer: heap-use-after-free on address 0x61e002a82a28 at pc 0x6d2f9d bp 0x7f1ff853ff70 sp 0x7f1ff853ff68
      READ of size 1855 at 0x61e002a82a28 thread T69
          #0 0x6d2f9c in __wt_buf_set /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/include/buf.i:78
          WT-1 0x6d25eb in __wt_session_copy_values /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/session/session_api.c:46
          WT-2 0x73fade in __wt_txn_begin /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/txn/txn.c:283
          WT-3 0xc2d85c in __wt_txn_autocommit_check /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/include/txn.i:192
          WT-4 0xc28f6e in __wt_page_in_func /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/btree/bt_page.c:128:15
          WT-5 0x964568 in __wt_page_swap_func /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/include/btree.i:1020
          WT-6 0x95e510 in __wt_row_search /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/btree/row_srch.c:295:17
          WT-7 0xb9967d in __cursor_row_search /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/btree/bt_cursor.c:238
          WT-8 0xb970a4 in __wt_btcur_search /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/btree/bt_cursor.c:309
          WT-9 0x9eafc7 in __curfile_search /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/cursor/cur_file.c:165
          WT-10 0x609cdd in __wt_metadata_search /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/meta/meta_table.c:200
          WT-11 0xb006c2 in __wt_meta_checkpoint_last_name /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/meta/meta_ckpt.c:72
          WT-12 0x704181 in __wt_session_get_btree_ckpt /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/session/session_dhandle.c:224
          WT-13 0x9fc62a in __wt_curfile_open /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/cursor/cur_file.c:453
          WT-14 0x6d4fcc in __wt_open_cursor /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/session/session_api.c:272
          WT-15 0xa96b72 in __clsm_open_cursors /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/lsm/lsm_cursor.c:573
          WT-16 0xabbb26 in __clsm_enter /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/lsm/lsm_cursor.c:225
          WT-17 0xab33dd in __clsm_remove /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/lsm/lsm_cursor.c:1393
          WT-18 0x4c1839 in row_remove /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/../../../test/format/ops.c:1041
          WT-19 0x4bb9ff in ops /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/../../../test/format/ops.c:383
          WT-20 0x7f200e02c181 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8181)
          WT-21 0x7f200d623fbc (/lib/x86_64-linux-gnu/libc.so.6+0xfafbc)
       
      0x61e002a82a28 is located 40 bytes inside of 2048-byte region [0x61e002a82a00,0x61e002a83200)
      freed by thread T69 here:
          #0 0x48b2f1 in __interceptor_free (/fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/t+0x48b2f1)
          WT-1 0x61e80b in __wt_free_int /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/os_posix/os_alloc.c:237
          WT-2 0x5756eb in __wt_buf_free /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/include/buf.i:112
          WT-3 0x5748d2 in __wt_cursor_close /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/cursor/cur_std.c:450
          WT-4 0x9f9d08 in __curfile_close /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/cursor/cur_file.c:328
          WT-5 0xab7ae9 in __clsm_close_cursors /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/lsm/lsm_cursor.c:349
          WT-6 0xa952f3 in __clsm_open_cursors /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/lsm/lsm_cursor.c:541
          WT-7 0xabbb26 in __clsm_enter /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/lsm/lsm_cursor.c:225
          WT-8 0xab33dd in __clsm_remove /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/lsm/lsm_cursor.c:1393
          WT-9 0x4c1839 in row_remove /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/../../../test/format/ops.c:1041
          WT-10 0x4bb9ff in ops /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/../../../test/format/ops.c:383
          WT-11 0x7f200e02c181 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8181)
       
      previously allocated by thread T69 here:
          #0 0x48bbbc in __interceptor_posix_memalign (/fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/t+0x48bbbc)
          WT-1 0x61d878 in __wt_realloc_aligned /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/os_posix/os_alloc.c:139
          WT-2 0x727877 in __wt_buf_grow_worker /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/support/scratch.c:44
          WT-3 0xcb61d2 in __wt_buf_grow /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/include/buf.i:17
          WT-4 0xcb5732 in __wt_buf_init /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/include/buf.i:46
          WT-5 0xcb0e88 in __wt_block_read_off /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/block/block_read.c:190
          WT-6 0xcb3add in __wt_bm_read /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/block/block_read.c:102
          WT-7 0xc0bebf in __wt_bt_read /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/btree/bt_io.c:34
          WT-8 0xc1ddf9 in __ovfl_read /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/btree/bt_ovfl.c:30
          WT-9 0xc1d3f1 in __wt_ovfl_read /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/btree/bt_ovfl.c:71
          WT-10 0xc5257c in __cell_data_ref /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/include/cell.i:748
          WT-11 0xc5156a in __wt_page_cell_data_ref /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/include/cell.i:789:10
          WT-12 0xc4ef9d in __wt_kv_return /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/btree/bt_ret.c:113
          WT-13 0xb97358 in __wt_btcur_search /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/btree/bt_cursor.c:313
          WT-14 0x9eafc7 in __curfile_search /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/cursor/cur_file.c:165
          WT-15 0xabda5b in __clsm_lookup /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/lsm/lsm_cursor.c:1041
          WT-16 0xaa60dd in __clsm_search /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/lsm/lsm_cursor.c:1088
          WT-17 0x4c175a in row_remove /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/../../../test/format/ops.c:1040
          WT-18 0x4bb9ff in ops /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/../../../test/format/ops.c:383
          WT-19 0x7f200e02c181 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8181)
       
      Thread T69 created by T0 here:
          #0 0x4776d5 in __interceptor_pthread_create (/fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/t+0x4776d5)
          WT-1 0x4b700e in wts_ops /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/../../../test/format/ops.c:98
          WT-2 0x4cddc1 in main /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/../../../test/format/t.c:190
          WT-3 0x7f200d54aec4 (/lib/x86_64-linux-gnu/libc.so.6+0x21ec4)
       
      SUMMARY: AddressSanitizer: heap-use-after-free /fast/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/include/buf.i:78 __wt_buf_set
      Shadow bytes around the buggy address:
        0x0c3c805484f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x0c3c80548500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x0c3c80548510: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x0c3c80548520: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x0c3c80548530: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      =>0x0c3c80548540: fd fd fd fd fd[fd]fd fd fd fd fd fd fd fd fd fd
        0x0c3c80548550: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x0c3c80548560: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x0c3c80548570: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x0c3c80548580: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x0c3c80548590: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
      Shadow byte legend (one shadow byte represents 8 application bytes):
        Addressable:           00
        Partially addressable: 01 02 03 04 05 06 07 
        Heap left redzone:       fa
        Heap right redzone:      fb
        Freed heap region:       fd
        Stack left redzone:      f1
        Stack mid redzone:       f2
        Stack right redzone:     f3
        Stack partial redzone:   f4
        Stack after return:      f5
        Stack use after scope:   f8
        Global redzone:          f9
        Global init order:       f6
        Poisoned by user:        f7
        Contiguous container OOB:fc
        ASan internal:           fe
      ==18828==ABORTING
      + cleanup
      + status=1
      + '[' -f RUNDIR/CONFIG ']'
      + cat RUNDIR/CONFIG
      ############################################
      #  RUN PARAMETERS
      ############################################
      auto_throttle=0
      firstfit=0
      bitcnt=1
      bloom=1
      bloom_bit_count=58
      bloom_hash_count=28
      bloom_oldest=0
      cache=300
      checkpoints=1
      checksum=uncompressed
      chunk_size=10
      compaction=0
      compression=none
      data_extend=0
      data_source=lsm
      delete_pct=8
      dictionary=0
      evict_max=1
      file_type=row-store
      backups=0
      huffman_key=0
      huffman_value=0
      insert_pct=84
      internal_key_truncation=1
      internal_page_max=12
      isolation=random
      key_gap=16
      key_max=32
      key_min=10
      leak_memory=0
      leaf_page_max=11
      logging=0
      lsm_worker_threads=3
      merge_max=13
      mmap=1
      ops=100000
      prefix_compression=0
      prefix_compression_min=6
      repeat_data_pct=20
      reverse=0
      rows=100000
      runs=1
      split_pct=78
      statistics=0
      threads=22
      value_max=2198
      value_min=12
      wiredtiger_config=
      write_pct=70
      ############################################
      

      Original failure:

      http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/2382

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                alexander.gorrod Alexander Gorrod
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: