Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-3067

Heap use after free on PPC test/format sanitizer build

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None

      Error:

      =================================================================
      ==5128==ERROR: AddressSanitizer: heap-use-after-free on address 0x0b90058f8800 at pc 0x0000100ccf34 bp 0x3fffa6d38ef0 sp 0x3fffa6d38f10
      WRITE of size 64 at 0x0b90058f8800 thread T19
          #0 0x100ccf30 in __asan_memcpy /home/mcahill/src/llvm-3.9.0.src/projects/compiler-rt/lib/asan/asan_interceptors.cc:413
          #1 0x10446c38 in __wt_bt_read /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/btree/bt_io.c:104:3
          #2 0x1045181c in __ovfl_read /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/btree/bt_ovfl.c:31:2
          #3 0x10451624 in __wt_ovfl_read /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/btree/bt_ovfl.c:73:6
          #4 0x1069cf00 in __cell_data_ref /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/include/cell.i:761:3
          #5 0x10699eb8 in __wt_page_cell_data_ref /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/include/cell.i:802:10
          #6 0x1069e2ec in __cursor_row_slot_return /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/include/cursor.i:402:10
          #7 0x1068f5dc in __cursor_row_prev /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/btree/bt_curprev.c:526:11
          #8 0x106898b4 in __wt_btcur_prev /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/btree/bt_curprev.c:605:11
          #9 0x10573e8c in __curfile_prev /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/cursor/cur_file.c:156:13
          #10 0x105f92d0 in __clsm_prev /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/lsm/lsm_cursor.c:1109:3
          #11 0x1014d9a0 in nextprev /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/test/format/../../../test/format/ops.c:931:47
          #12 0x101465bc in ops /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/test/format/../../../test/format/ops.c:707:16
          #13 0x101001c8 in __asan::AsanThread::ThreadStart(unsigned long, __sanitizer::atomic_uintptr_t*) /home/mcahill/src/llvm-3.9.0.src/projects/compiler-rt/lib/asan/asan_thread.cc:250
          #14 0x10032198 in asan_thread_start(void*) /home/mcahill/src/llvm-3.9.0.src/projects/compiler-rt/lib/asan/asan_interceptors.cc:236
          #15 0x3fffb0288940 in start_thread (/lib64/power8/libpthread.so.0+0x8940)
          #16 0x3fffaffd763c in __clone (/lib64/power8/libc.so.6+0x11763c)
      
      0x0b90058f8800 is located 0 bytes inside of 86594-byte region [0x0b90058f8800,0x0b900590da42)
      freed by thread T27 here:
          #0 0x100ea7fc in __interceptor_cfree.localalias.0 /home/mcahill/src/llvm-3.9.0.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:54
          #1 0x10273fb8 in __wt_free_int /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/os_common/os_alloc.c:309:2
          #2 0x101d6ec4 in __wt_buf_free /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/include/buf.i:102:2
          #3 0x101d6a84 in __wt_cursor_close /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/cursor/cur_std.c:554:2
          #4 0x1057d06c in __curfile_close /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/cursor/cur_file.c:375:2
          #5 0x105f3c34 in __clsm_close_cursors /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/lsm/lsm_cursor.c:393:4
          #6 0x105f34d0 in __wt_clsm_close /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/lsm/lsm_cursor.c:1627:2
          #7 0x10338218 in __session_close /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/session/session_api.c:177:3
          #8 0x10143984 in ops /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/test/format/../../../test/format/ops.c:449:5
          #9 0x101001c8 in __asan::AsanThread::ThreadStart(unsigned long, __sanitizer::atomic_uintptr_t*) /home/mcahill/src/llvm-3.9.0.src/projects/compiler-rt/lib/asan/asan_thread.cc:250
          #10 0x10032198 in asan_thread_start(void*) /home/mcahill/src/llvm-3.9.0.src/projects/compiler-rt/lib/asan/asan_interceptors.cc:236
          #11 0x3fffb0288940 in start_thread (/lib64/power8/libpthread.so.0+0x8940)
          #12 0x3fffaffd763c in __clone (/lib64/power8/libc.so.6+0x11763c)
      
      previously allocated by thread T19 here:
          #0 0x100eaf3c in realloc /home/mcahill/src/llvm-3.9.0.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:77
          #1 0x10272e70 in __realloc_func /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/os_common/os_alloc.c:130:11
          #2 0x10273030 in __wt_realloc_noclear /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/os_common/os_alloc.c:171:10
          #3 0x10382c64 in __wt_buf_grow_worker /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/support/scratch.c:48:4
          #4 0x1044ce3c in __wt_buf_grow /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/include/buf.i:18:6
          #5 0x104486a4 in __wt_buf_initsize /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/include/buf.i:61:2
          #6 0x10446b84 in __wt_bt_read /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/btree/bt_io.c:92:3
          #7 0x1045181c in __ovfl_read /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/btree/bt_ovfl.c:31:2
          #8 0x10451624 in __wt_ovfl_read /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/btree/bt_ovfl.c:73:6
          #9 0x1069cf00 in __cell_data_ref /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/include/cell.i:761:3
          #10 0x10699eb8 in __wt_page_cell_data_ref /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/include/cell.i:802:10
          #11 0x1069e2ec in __cursor_row_slot_return /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/include/cursor.i:402:10
          #12 0x1068f5dc in __cursor_row_prev /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/btree/bt_curprev.c:526:11
          #13 0x106898b4 in __wt_btcur_prev /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/btree/bt_curprev.c:605:11
          #14 0x10573e8c in __curfile_prev /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/cursor/cur_file.c:156:13
          #15 0x105f92d0 in __clsm_prev /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/../src/lsm/lsm_cursor.c:1109:3
          #16 0x1014d9a0 in nextprev /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/test/format/../../../test/format/ops.c:931:47
          #17 0x101465bc in ops /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/test/format/../../../test/format/ops.c:707:16
          #18 0x101001c8 in __asan::AsanThread::ThreadStart(unsigned long, __sanitizer::atomic_uintptr_t*) /home/mcahill/src/llvm-3.9.0.src/projects/compiler-rt/lib/asan/asan_thread.cc:250
          #19 0x10032198 in asan_thread_start(void*) /home/mcahill/src/llvm-3.9.0.src/projects/compiler-rt/lib/asan/asan_interceptors.cc:236
          #20 0x3fffb0288940 in start_thread (/lib64/power8/libpthread.so.0+0x8940)
          #21 0x3fffaffd763c in __clone (/lib64/power8/libc.so.6+0x11763c)
      
      Thread T19 created by T0 here:
          #0 0x10031f28 in pthread_create /home/mcahill/src/llvm-3.9.0.src/projects/compiler-rt/lib/asan/asan_interceptors.cc:245
          #1 0x101416cc in wts_ops /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/test/format/../../../test/format/ops.c:110:3
          #2 0x10155440 in main /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/test/format/../../../test/format/t.c:216:5
          #3 0x3fffafee457c in generic_start_main.isra.0 (/lib64/power8/libc.so.6+0x2457c)
          #4 0x3fffafee4770 in __libc_start_main (/lib64/power8/libc.so.6+0x24770)
      
      Thread T27 created by T0 here:
          #0 0x10031f28 in pthread_create /home/mcahill/src/llvm-3.9.0.src/projects/compiler-rt/lib/asan/asan_interceptors.cc:245
          #1 0x101416cc in wts_ops /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/test/format/../../../test/format/ops.c:110:3
          #2 0x10155440 in main /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer-ppc/build_posix/test/format/../../../test/format/t.c:216:5
          #3 0x3fffafee457c in generic_start_main.isra.0 (/lib64/power8/libc.so.6+0x2457c)
          #4 0x3fffafee4770 in __libc_start_main (/lib64/power8/libc.so.6+0x24770)
      
      SUMMARY: AddressSanitizer: heap-use-after-free /home/mcahill/src/llvm-3.9.0.src/projects/compiler-rt/lib/asan/asan_interceptors.cc:413 in __asan_memcpy
      Shadow bytes around the buggy address:
        0x037200b1f0b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x037200b1f0c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x037200b1f0d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x037200b1f0e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x037200b1f0f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      =>0x037200b1f100:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x037200b1f110: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x037200b1f120: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x037200b1f130: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x037200b1f140: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x037200b1f150: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
      Shadow byte legend (one shadow byte represents 8 application bytes):
        Addressable:           00
        Partially addressable: 01 02 03 04 05 06 07 
        Heap left redzone:       fa
        Heap right redzone:      fb
        Freed heap region:       fd
        Stack left redzone:      f1
        Stack mid redzone:       f2
        Stack right redzone:     f3
        Stack partial redzone:   f4
        Stack after return:      f5
        Stack use after scope:   f8
        Global redzone:          f9
        Global init order:       f6
        Poisoned by user:        f7
        Container overflow:      fc
        Array cookie:            ac
        Intra object redzone:    bb
        ASan internal:           fe
        Left alloca redzone:     ca
        Right alloca redzone:    cb
      ==5128==ABORTING
      

      Config

      ############################################
      #  RUN PARAMETERS
      ############################################
      abort=0
      auto_throttle=1
      backups=0
      bitcnt=2
      bloom=1
      bloom_bit_count=55
      bloom_hash_count=6
      bloom_oldest=0
      cache=270
      checkpoints=1
      checksum=uncompressed
      chunk_size=9
      compaction=0
      compression=zlib
      data_extend=0
      data_source=lsm
      delete_pct=0
      dictionary=0
      direct_io=0
      encryption=none
      evict_max=0
      file_type=row-store
      firstfit=0
      huffman_key=0
      huffman_value=0
      in_memory=0
      insert_pct=53
      internal_key_truncation=1
      internal_page_max=17
      isolation=read-committed
      key_gap=17
      key_max=93
      key_min=18
      leaf_page_max=11
      leak_memory=0
      logging=0
      logging_archive=1
      logging_compression=none
      logging_prealloc=1
      long_running_txn=0
      lsm_worker_threads=3
      merge_max=7
      mmap=1
      ops=100000
      prefix_compression=1
      prefix_compression_min=0
      quiet=1
      repeat_data_pct=47
      reverse=0
      rows=100000
      runs=1
      rebalance=1
      salvage=1
      split_pct=73
      statistics=0
      statistics_server=0
      threads=13
      timer=20
      transaction-frequency=74
      value_max=3478
      value_min=4
      verify=1
      wiredtiger_config=
      write_pct=83
      ############################################
      

            Assignee:
            backlog-server-execution [DO NOT USE] Backlog - Storage Execution Team
            Reporter:
            david.hows David Hows
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: