Loading...

XML

Word

Printable

JSON

Type: Task
Resolution: Done
Fix Version/s: WT2.2
Affects Version/s: None
Component/s: None
Labels:
- Bug

@michaelcahill, I've stared at this one, but I don't see it.

One thread is looking at page->type:

    733         parent_decr += size;
    734         WT_TRET(__wt_session_fotxn_add(session, pindex, size));
    735         switch (parent->type) {
    736         case WT_PAGE_ROW_INT:
    737         case WT_PAGE_ROW_LEAF:
    738                 if ((ikey = __wt_ref_key_instantiated(ref)) == NULL)
    739                         break;
    740                 size = sizeof(WT_IKEY) + ikey->size;
    741                 parent_decr += size;
    742                 WT_TRET(__wt_session_fotxn_add(session, ikey, size));
    743                 break;
    744         }

and the page has been evicted/freed.

I want to blame the code that loops until we get a matching ref->home and parent, but I don't see any problems there.

We looked at page->type 20 lines before, line 713, so this must have happened simultaneously, so somehow our parent got evicted while we were splitting into it.

Is there some way an internal page could be selected for eviction after a split but before the child pages appear on it? I don't see how that's possible, either.

=================================================================
==105189==ERROR: AddressSanitizer: heap-use-after-free on address 0x607000bc18f0 at pc 0x4febbf bp 0x7f64b0cdbc50 sp 0x7f64b0cdbc48
READ of size 1 at 0x607000bc18f0 thread T79
    #0 0x4febbe in __wt_split_evict <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/rec_split.c>:735
    WT-1 0x4fb3e3 in __rec_page_dirty_update <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/rec_evict.c>:172
    WT-2 0x4fb3e3 in __wt_rec_evict <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/rec_evict.c>:88
    WT-3 0x4d1bdd in __wt_evict_page <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/bt_evict.c>:393
    WT-4 0x4d1bdd in __wt_evict_lru_page <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/bt_evict.c>:977
    WT-5 0x599585 in __wt_cache_full_check <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/include/cache.i>:93
    WT-6 0x599585 in __cursor_enter <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/include/cursor.i>:57
    WT-7 0x599585 in __curfile_enter <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/include/cursor.i>:94
    WT-8 0x599585 in __cursor_func_init <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/include/cursor.i>:141
    WT-9 0x59cc17 in __wt_btcur_update <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/bt_cursor.c>:522:38
    WT-10 0x547062 in __curfile_update <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/cursor/cur_file.c>:262
    WT-11 0x452fd7 in row_update <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/test/format/../../../test/format/ops.c>:701
    WT-12 0x452fd7 in ops <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/test/format/../../../test/format/ops.c>:372
    WT-13 0x4415b3 in __asan::AsanThread::ThreadStart(unsigned long) (<http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/test/format/t+0x4415b3)>
    WT-14 0x339e807c52 in start_thread (/lib64/libpthread.so.0+0x339e807c52)
    WT-15 0x339e0f5dbc in __clone (/lib64/libc.so.6+0x339e0f5dbc)
0x607000bc18f0 is located 64 bytes inside of 72-byte region [0x607000bc18b0,0x607000bc18f8)
freed by thread T66 here:
    #0 0x43af54 in free (<http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/test/format/t+0x43af54)>
    WT-1 0x5a50f3 in __wt_page_out <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/bt_discard.c>:97
    WT-2 0x4fb738 in __rec_page_dirty_update <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/rec_evict.c>:192
    WT-3 0x4fb738 in __wt_rec_evict <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/rec_evict.c>:88
    WT-4 0x4d1bdd in __wt_evict_page <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/bt_evict.c>:393
    WT-5 0x4d1bdd in __wt_evict_lru_page <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/bt_evict.c>:977
previously allocated by thread T78 here:
    #0 0x43b0a3 in calloc (<http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/test/format/t+0x43b0a3)>
    WT-1 0x4889cc in __wt_calloc <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/os_posix/os_alloc.c>:38
    WT-2 0x4db993 in __wt_page_inmem <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/bt_page.c>:294
    WT-3 0x4debe6 in __wt_cache_read <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/bt_read.c>:109
    WT-4 0x4da997 in __wt_page_in_func <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/bt_page.c>:46
    WT-5 0x52c51e in __wt_page_swap_func <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/include/btree.i>:704
    WT-6 0x52c51e in __wt_row_search <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/row_srch.c>:278
    WT-7 0x59b713 in __cursor_row_search <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/bt_cursor.c>:158
    WT-8 0x59b713 in __wt_btcur_insert <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/bt_cursor.c>:393
    WT-9 0x5464c7 in __curfile_insert <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/cursor/cur_file.c>:223
Thread T79 created by T0 here:
    #0 0x436fa0 in __interceptor_pthread_create (<http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/test/format/t+0x436fa0)>
    WT-1 0x450949 in wts_ops <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/test/format/../../../test/format/ops.c>:94
    WT-2 0x456992 in main <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/test/format/../../../test/format/t.c>:180
Thread T66 created by T0 here:
    #0 0x436fa0 in __interceptor_pthread_create (<http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/test/format/t+0x436fa0)>
    WT-1 0x450949 in wts_ops <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/test/format/../../../test/format/ops.c>:94
    WT-2 0x456992 in main <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/test/format/../../../test/format/t.c>:180
Thread T78 created by T0 here:
    #0 0x436fa0 in __interceptor_pthread_create (<http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/test/format/t+0x436fa0)>
    WT-1 0x450949 in wts_ops <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/test/format/../../../test/format/ops.c>:94
    WT-2 0x456992 in main <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/test/format/../../../test/format/t.c>:180
SUMMARY: AddressSanitizer: heap-use-after-free <http://mjc.homeunix.org:8180/job/wiredtiger-test-format-stress-santizer/ws/build_posix/../src/btree/rec_split.c>:735 __wt_split_evict
Shadow bytes around the buggy address:
  0x0c0e801702c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0e801702d0: fd fd fd fd fd fd fd fd fd fd fa fa fa fa fd fd
  0x0c0e801702e0: fd fd fd fd fd fd fd fa fa fa fa fa fd fd fd fd
  0x0c0e801702f0: fd fd fd fd fd fd fa fa fa fa fa fa fa fa fa fa
  0x0c0e80170300: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
=>0x0c0e80170310: fd fa fa fa fa fa fd fd fd fd fd fd fd fd[fd]fa
  0x0c0e80170320: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0e80170330: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0e80170340: fd fd fd fd fd fd fd fd fd fd fa fa fa fa fd fd
  0x0c0e80170350: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
  0x0c0e80170360: fd fd fd fd fd fd fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:     fa
  Heap right redzone:    fb
  Freed heap region:     fd
  Stack left redzone:    f1
  Stack mid redzone:     f2
  Stack right redzone:   f3
  Stack partial redzone: f4
  Stack after return:    f5
  Stack use after scope: f8
  Global redzone:        f9
  Global init order:     f6
  Poisoned by user:      f7
  ASan internal:         fe
==105189==ABORTING
+ status=1
+ '[' -f RUNDIR/run ']'
+ cat RUNDIR/run
############################################
#  RUN PARAMETERS
############################################
auto_throttle=1
firstfit=1
# bitcnt not applicable to this run
bloom=1
bloom_bit_count=48
bloom_hash_count=7
bloom_oldest=0
cache=3
checkpoints=1
checksum=off
chunk_size=7
compaction=0
compression=none
data_extend=0
data_source=file
delete_pct=12
dictionary=0
file_type=row-store
hot_backups=0
huffman_key=0
huffman_value=0
insert_pct=18
internal_key_truncation=1
internal_page_max=14
key_gap=1
key_max=72
key_min=25
leaf_page_max=10
merge_max=19
merge_threads=1
mmap=1
ops=100000
prefix_compression=0
prefix_compression_min=8
repeat_data_pct=7
reverse=0
rows=100000
runs=1
split_pct=58
statistics=0
threads=30
value_max=814
value_min=19
# wiredtiger_config not applicable to this run
write_pct=59
############################################

related to

WT-1 placeholder WT-1

Closed

WT-2 What does metadata look like?

Closed

WT-3 What file formats are required?

Closed

WT-4 Flexible cursor traversals

Closed

WT-5 How does pget work: is it necessary?

Closed

WT-6 Complex schema example

Closed

WT-7 Do we need the handle->err/errx methods?

Closed

WT-8 Do we need table load, bulk-load and/or dump methods?

Closed

WT-9 Does adding schema need to be transactional?

Closed

WT-10 Basic "getting started" tutorial

Closed

WT-11 placeholder #11

Closed

WT-12 Write more examples

Closed

WT-13 Define supported platforms

Closed

WT-14 Windows build

Closed

WT-15 Automated build/test infrastructure

Closed

(10 related to)

Assignee:: Michael Cahill (Inactive)

Reporter:: Keith Bostic (Inactive)

Votes:: 0 Vote for this issue

Watchers:: 0 Start watching this issue

Created:: Apr 08 2014 03:48:10 PM UTC

Updated:: Apr 16 2015 07:23:55 PM UTC

Resolved:: Apr 09 2015 01:07:59 AM UTC

Details

Description

Attachments

Issue Links

Activity

People

Dates