Major - P3 The Jenkins job failed for wiredtiger-test-race-condition-stress-sanitizer with AddressSanitizer complaining of heap-buffer-overflow:
http://build.wiredtiger.com:8080/job/wiredtiger-test-race-condition-stress-sanitizer/15116/console
############################################ # RUN PARAMETERS ############################################ abort=0 alter=0 auto_throttle=1 backups=0 bitcnt=4 bloom=1 bloom_bit_count=43 bloom_hash_count=16 bloom_oldest=0 cache=37 cache_minimum=20 checkpoints=off checkpoint_log_size=21 checkpoint_wait=86 checksum=uncompressed chunk_size=8 compaction=0 compression=none data_extend=0 data_source=file delete_pct=80 dictionary=0 direct_io=0 encryption=none evict_max=3 file_type=row-store firstfit=0 huffman_key=0 huffman_value=0 independent_thread_rng=1 in_memory=1 insert_pct=1 internal_key_truncation=1 internal_page_max=15 isolation=random key_gap=10 key_max=32 key_min=29 leaf_page_max=9 leak_memory=0 logging=0 logging_archive=1 logging_compression=none logging_file_max=12507 logging_prealloc=1 long_running_txn=0 lsm_worker_threads=3 merge_max=4 mmap=1 modify_pct=2 ops=100000 prefix_compression=1 prefix_compression_min=6 quiet=1 read_pct=2 rebalance=0 repeat_data_pct=68 reverse=0 rows=100000 runs=1 salvage=0 split_pct=54 statistics=0 statistics_server=0 threads=27 timer=360 transaction_timestamps=0 transaction-frequency=62 value_max=80 value_min=3 verify=0 wiredtiger_config= write_pct=15 ############################################
The Error:
==40111==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60700015a790 at pc 0x000000869ff5 bp 0x7f6bb2778cc0 sp 0x7f6bb2778cb8
READ of size 8 at 0x60700015a790 thread T58
#0 0x869ff4 in __wt_row_leaf_key /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/include/btree.i:1001:9
#1 0x865177 in __key_return /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/btree/bt_ret.c:62:11
#2 0x8649ff in __wt_key_return /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/btree/bt_ret.c:267:3
#3 0xaf6da7 in __wt_btcur_remove /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/btree/bt_cursor.c:1089:4
#4 0x97dffa in __curfile_remove /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/cursor/cur_file.c:398:2
#5 0x52c2a3 in row_remove /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/test/format/../../../test/format/ops.c:1698:9
#6 0x527196 in ops /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/test/format/../../../test/format/ops.c:808:9
#7 0x7f6bc7067dc4 in start_thread /usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:308
#8 0x7f6bc624e76c in __clone (/lib64/libc.so.6+0xf776c)
0x60700015a790 is located 0 bytes to the right of 80-byte region [0x60700015a740,0x60700015a790)
allocated by thread T58 here:
#0 0x4dfa6d in calloc /home/bostic/src/llvm40/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:74
#1 0x6330b2 in __wt_calloc /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/os_common/os_alloc.c:52:11
#2 0x836ee9 in __wt_page_alloc /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/btree/bt_page.c:63:2
#3 0x81ae8c in __wt_btree_new_leaf_page /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/btree/bt_handle.c:694:3
#4 0x84c94c in __page_read /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/btree/bt_read.c:370:3
#5 0x84a10c in __wt_page_in_func /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/btree/bt_read.c:592:4
#6 0x900827 in __wt_page_swap_func /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/include/btree.i:1512:8
#7 0x8fc2e8 in __wt_row_search /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/btree/row_srch.c:446:14
#8 0xaf06a1 in __cursor_row_search /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/btree/bt_cursor.c:377:2
#9 0xaf6435 in __wt_btcur_remove /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/btree/bt_cursor.c:1015:3
#10 0x97dffa in __curfile_remove /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/cursor/cur_file.c:398:2
#11 0x52c2a3 in row_remove /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/test/format/../../../test/format/ops.c:1698:9
#12 0x527196 in ops /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/test/format/../../../test/format/ops.c:808:9
#13 0x7f6bc7067dc4 in start_thread /usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:308
Thread T58 created by T0 here:
#0 0x4373c1 in __interceptor_pthread_create /home/bostic/src/llvm40/projects/compiler-rt/lib/asan/asan_interceptors.cc:305
#1 0x64d040 in __wt_thread_create /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/os_posix/os_thread.c:30:2
#2 0x522a3b in wts_ops /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/test/format/../../../test/format/ops.c:164:3
#3 0x5334dd in main /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/test/format/../../../test/format/t.c:209:5
#4 0x7f6bc6178b34 in __libc_start_main /usr/src/debug/glibc-2.17-c758a686/csu/../csu/libc-start.c:274
SUMMARY: AddressSanitizer: heap-buffer-overflow /mnt/data0/jenkins/workspace/wiredtiger-test-race-condition-stress-sanitizer/build_posix/../src/include/btree.i:1001:9 in __wt_row_leaf_key
Shadow bytes around the buggy address:
0x0c0e800234a0: fa fa 00 00 00 00 00 00 00 00 00 07 fa fa fa fa
0x0c0e800234b0: 00 00 00 00 00 00 00 00 00 00 fa fa fa fa 00 00
0x0c0e800234c0: 00 00 00 00 00 00 00 00 fa fa fa fa 00 00 00 00
0x0c0e800234d0: 00 00 00 00 00 00 fa fa fa fa 00 00 00 00 00 00
0x0c0e800234e0: 00 00 06 fa fa fa fa fa 00 00 00 00 00 00 00 00
=>0x0c0e800234f0: 00 00[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c0e80023500: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fa fa
0x0c0e80023510: fa fa 00 00 00 00 00 00 00 00 00 00 fa fa fa fa
0x0c0e80023520: fd fd fd fd fd fd fd fd fd fd fa fa fa fa 00 00
0x0c0e80023530: 00 00 00 00 00 00 00 00 fa fa fa fa 00 00 00 00
0x0c0e80023540: 00 00 00 00 00 00 fa fa fa fa 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==40111==ABORTING