Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2371

parent split cannot access the page after page-index swap

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.8.0
    • Labels:
      None
    • # Replies:
      8
    • Last comment by Customer:
      true

      Description

      Clang reports:

        60: file, row-store (611 seconds)
      =================================================================
      ==23631==ERROR: AddressSanitizer: heap-use-after-free on address 0x60c00001fe60 at pc 0x0000006dfe4e bp 0x7f2c67f97d50 sp
       0x7f2c67f97d48
      READ of size 1 at 0x60c00001fe60 thread T3147
          #0 0x6dfe4d in __split_parent /home/ec2-user/work/wiredtiger/src/btree/bt_split.c:880:2
          #1 0x6d7d17 in __split_insert /home/ec2-user/work/wiredtiger/src/btree/bt_split.c:1930:13
          #2 0x6d7d17 in __wt_split_insert /home/ec2-user/work/wiredtiger/src/btree/bt_split.c:2003
          #3 0x546b2d in __evict_review /home/ec2-user/work/wiredtiger/src/evict/evict_page.c:436:12
          #4 0x546b2d in __wt_evict /home/ec2-user/work/wiredtiger/src/evict/evict_page.c:81
          #5 0x542d60 in __evict_page /home/ec2-user/work/wiredtiger/src/evict/evict_lru.c:1488:2
          #6 0x5419ad in __wt_cache_eviction_worker /home/ec2-user/work/wiredtiger/src/evict/evict_lru.c:1567:17
          #7 0x7c749c in __wt_cache_eviction_check /home/ec2-user/work/wiredtiger/./src/include/cache.i:245:10
          #8 0x7c749c in __cursor_enter /home/ec2-user/work/wiredtiger/./src/include/cursor.i:59
          #9 0x7c749c in __curfile_enter /home/ec2-user/work/wiredtiger/./src/include/cursor.i:93
          #10 0x7c749c in __cursor_func_init /home/ec2-user/work/wiredtiger/./src/include/cursor.i:266
          #11 0x7c56f6 in __wt_btcur_search /home/ec2-user/work/wiredtiger/src/btree/bt_cursor.c:335:3
          #12 0x7429bd in __curfile_search /home/ec2-user/work/wiredtiger/src/cursor/cur_file.c:200:2
          #13 0x4f064b in row_remove /home/ec2-user/work/wiredtiger/test/format/ops.c:1155:13
          #14 0x4f064b in ops /home/ec2-user/work/wiredtiger/test/format/ops.c:437
          #15 0x7f2c7dee2dc4 in start_thread (/lib64/libpthread.so.0+0x7dc4)
          #16 0x7f2c7d0c7bdc in __clone (/lib64/libc.so.6+0xf6bdc)
       
      0x60c00001fe60 is located 32 bytes inside of 120-byte region [0x60c00001fe40,0x60c00001feb8)
      freed by thread T3150 here:
          #0 0x4c6d92 in free (/mnt/fast/bostic/work/wiredtiger/test/format/t+0x4c6d92)
          #1 0x6a12cf in __wt_page_out /home/ec2-user/work/wiredtiger/src/btree/bt_discard.c:139:2
          #2 0x546174 in __evict_page_dirty_update /home/ec2-user/work/wiredtiger/src/evict/evict_page.c:315:3
          #3 0x546174 in __wt_evict /home/ec2-user/work/wiredtiger/src/evict/evict_page.c:124
          #4 0x542d60 in __evict_page /home/ec2-user/work/wiredtiger/src/evict/evict_lru.c:1488:2
          #5 0x5419ad in __wt_cache_eviction_worker /home/ec2-user/work/wiredtiger/src/evict/evict_lru.c:1567:17
          #6 0x7c749c in __wt_cache_eviction_check /home/ec2-user/work/wiredtiger/./src/include/cache.i:245:10
          #7 0x7c749c in __cursor_enter /home/ec2-user/work/wiredtiger/./src/include/cursor.i:59
          #8 0x7c749c in __curfile_enter /home/ec2-user/work/wiredtiger/./src/include/cursor.i:93
          #9 0x7c749c in __cursor_func_init /home/ec2-user/work/wiredtiger/./src/include/cursor.i:266
          #10 0x7c7edd in __wt_btcur_search_near /home/ec2-user/work/wiredtiger/src/btree/bt_cursor.c:425:3
          #11 0x74320b in __curfile_search_near /home/ec2-user/work/wiredtiger/src/cursor/cur_file.c:222:2
          #12 0x4f3d22 in read_row /home/ec2-user/work/wiredtiger/test/format/ops.c:645:9
          #13 0x4f04f3 in ops /home/ec2-user/work/wiredtiger/test/format/ops.c:494:8
          #14 0x7f2c7dee2dc4 in start_thread (/lib64/libpthread.so.0+0x7dc4)
       
      previously allocated by thread T3121 here:
          #0 0x4c71eb in calloc (/mnt/fast/bostic/work/wiredtiger/test/format/t+0x4c71eb)
          #1 0x57a35b in __wt_calloc /home/ec2-user/work/wiredtiger/src/os_posix/os_alloc.c:60:11
          #2 0x6b1014 in __wt_page_alloc /home/ec2-user/work/wiredtiger/src/btree/bt_page.c:63:2
          #3 0x6b1d55 in __wt_page_inmem /home/ec2-user/work/wiredtiger/src/btree/bt_page.c:194:2
          #4 0x6b8711 in __page_read /home/ec2-user/work/wiredtiger/src/btree/bt_read.c:394:2
          #5 0x6b8711 in __wt_page_in_func /home/ec2-user/work/wiredtiger/src/btree/bt_read.c:488
          #6 0x70055f in __wt_page_swap_func /home/ec2-user/work/wiredtiger/./src/include/btree.i:1310:8
          #7 0x70055f in __tree_walk_internal /home/ec2-user/work/wiredtiger/src/btree/bt_walk.c:504
          #8 0x7b5378 in __wt_btcur_next /home/ec2-user/work/wiredtiger/src/btree/bt_curnext.c:666:3
          #9 0x741dbe in __curfile_next /home/ec2-user/work/wiredtiger/src/cursor/cur_file.c:113:13
          #10 0x72224c in __wt_las_sweep /home/ec2-user/work/wiredtiger/src/cache/cache_las.c:334:27
          #11 0x5244ad in __sweep_server /home/ec2-user/work/wiredtiger/src/conn/conn_sweep.c:285:4
          #12 0x7f2c7dee2dc4 in start_thread (/lib64/libpthread.so.0+0x7dc4)
      

      The problem is we've swapped the new page-index into place in the parent, after which eviction finds and evicts the page, after which the split code references the page->type field.

        Issue Links

          Activity

          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith@wiredtiger.com'}

          Message: WT-2371: parent split cannot access the page after page-index swap

          After swapping the new page-index into place in the parent, we can no
          longer look at the contents of the page, it may have already been evicted.
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/d55ad9ab7bebc4e0f42b8bd3126b3b93849e4ae1

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith@wiredtiger.com'} Message: WT-2371 : parent split cannot access the page after page-index swap After swapping the new page-index into place in the parent, we can no longer look at the contents of the page, it may have already been evicted. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/d55ad9ab7bebc4e0f42b8bd3126b3b93849e4ae1
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: Merge pull request #2471 from wiredtiger/wt-2371

          WT-2371: parent split cannot access the page after page-index swap
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/7fa9fcd0ace4540d8048b1e5626c30d03f098730

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Merge pull request #2471 from wiredtiger/wt-2371 WT-2371 : parent split cannot access the page after page-index swap Branch: develop https://github.com/wiredtiger/wiredtiger/commit/7fa9fcd0ace4540d8048b1e5626c30d03f098730
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexg@wiredtiger.com'}

          Message: Import wiredtiger-wiredtiger-2.7.0-592-gcc96d99.tar.gz from wiredtiger branch mongodb-3.4

          ref: 7fea169..cc96d99

          WT-60 Big endian port
          WT-2130 Improve on-disk page utlilization with random workloads
          WT-2215 WT_LSN needs to support atomic reads and updates
          WT-2295 WT_SESSION.create does a full-scan of the main table
          WT-2352 Allow build and test without requiring lz4
          WT-2356 Log scan advances to next log file on partially written record
          WT-2363 Remove built in support for bzip2
          WT-2368 Row-store can pass garbage keys to collator functions
          WT-2369 Use C compiler to detect headers instead of C++ compiler
          WT-2371 Parent split cannot access the page after page-index swap
          WT-2372 WiredTiger windows builder fails with C4005 against the "inline" macro
          WT-2377 WTPERF doesn't compile in Windows under MSVC
          WT-2378 Tasks time out on LSM builder
          SERVER-22388 WiredTiger changes for MongoDB 3.3.2
          SERVER-22437 Coverity analysis defect 77704: Redundant test
          SERVER-22438 Coverity analysis defect 77705: Dereference before null check
          Branch: master
          https://github.com/mongodb/mongo/commit/79a378bddf664714190dc9d0a34b1d8cd4951588

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexg@wiredtiger.com'} Message: Import wiredtiger-wiredtiger-2.7.0-592-gcc96d99.tar.gz from wiredtiger branch mongodb-3.4 ref: 7fea169..cc96d99 WT-60 Big endian port WT-2130 Improve on-disk page utlilization with random workloads WT-2215 WT_LSN needs to support atomic reads and updates WT-2295 WT_SESSION.create does a full-scan of the main table WT-2352 Allow build and test without requiring lz4 WT-2356 Log scan advances to next log file on partially written record WT-2363 Remove built in support for bzip2 WT-2368 Row-store can pass garbage keys to collator functions WT-2369 Use C compiler to detect headers instead of C++ compiler WT-2371 Parent split cannot access the page after page-index swap WT-2372 WiredTiger windows builder fails with C4005 against the "inline" macro WT-2377 WTPERF doesn't compile in Windows under MSVC WT-2378 Tasks time out on LSM builder SERVER-22388 WiredTiger changes for MongoDB 3.3.2 SERVER-22437 Coverity analysis defect 77704: Redundant test SERVER-22438 Coverity analysis defect 77705: Dereference before null check Branch: master https://github.com/mongodb/mongo/commit/79a378bddf664714190dc9d0a34b1d8cd4951588
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith@wiredtiger.com'}

          Message: WT-2371: parent split cannot access the page after page-index swap

          After swapping the new page-index into place in the parent, we can no
          longer look at the contents of the page, it may have already been evicted.
          Branch: mongodb-3.2
          https://github.com/wiredtiger/wiredtiger/commit/d55ad9ab7bebc4e0f42b8bd3126b3b93849e4ae1

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith@wiredtiger.com'} Message: WT-2371 : parent split cannot access the page after page-index swap After swapping the new page-index into place in the parent, we can no longer look at the contents of the page, it may have already been evicted. Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/d55ad9ab7bebc4e0f42b8bd3126b3b93849e4ae1
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: Merge pull request #2471 from wiredtiger/wt-2371

          WT-2371: parent split cannot access the page after page-index swap
          Branch: mongodb-3.2
          https://github.com/wiredtiger/wiredtiger/commit/7fa9fcd0ace4540d8048b1e5626c30d03f098730

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Merge pull request #2471 from wiredtiger/wt-2371 WT-2371 : parent split cannot access the page after page-index swap Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/7fa9fcd0ace4540d8048b1e5626c30d03f098730
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith@wiredtiger.com'}

          Message: WT-2371: parent split cannot access the page after page-index swap

          After swapping the new page-index into place in the parent, we can no
          longer look at the contents of the page, it may have already been evicted.
          Branch: mongodb-3.2
          https://github.com/wiredtiger/wiredtiger/commit/d55ad9ab7bebc4e0f42b8bd3126b3b93849e4ae1

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith@wiredtiger.com'} Message: WT-2371 : parent split cannot access the page after page-index swap After swapping the new page-index into place in the parent, we can no longer look at the contents of the page, it may have already been evicted. Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/d55ad9ab7bebc4e0f42b8bd3126b3b93849e4ae1
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: Merge pull request #2471 from wiredtiger/wt-2371

          WT-2371: parent split cannot access the page after page-index swap
          Branch: mongodb-3.2
          https://github.com/wiredtiger/wiredtiger/commit/7fa9fcd0ace4540d8048b1e5626c30d03f098730

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Merge pull request #2471 from wiredtiger/wt-2371 WT-2371 : parent split cannot access the page after page-index swap Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/7fa9fcd0ace4540d8048b1e5626c30d03f098730
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}

          Message: Import wiredtiger-wiredtiger-2.7.0-650-g5cdd3e3.tar.gz from wiredtiger branch mongodb-3.2

          ref: 07966a4..5cdd3e3

          SERVER-22437 Coverity analysis defect 77704: Redundant test
          SERVER-22438 Coverity analysis defect 77705: Dereference before null check
          SERVER-22676 WiredTiger fails to open databases created by 3.0.0 or 3.0.1
          WT-2130 Improve on-disk page utlilization with random workloads
          WT-2215 WT_LSN needs to support atomic reads and updates
          WT-2295 WT_SESSION.create does a full-scan of the main table
          WT-2352 Allow build and test without requiring lz4
          WT-2356 log scan advances to next log file on partially written record
          WT-2363 Remove built in support for bzip2
          WT-2368 row-store can pass garbage keys to collator functions
          WT-2369 Use C compiler to detect headers instead of C++ compiler
          WT-2371 parent split cannot access the page after page-index swap
          WT-2372 WiredTiger windows builder fails with C4005 against the "inline" macro
          WT-2377 WTPERF doesn't compile in Windows under MSVC
          WT-2378 Tasks time out on LSM builder
          WT-2397 Cursor traversal from end of the tree skips records.
          WT-60 Big endian port
          Branch: v3.2
          https://github.com/mongodb/mongo/commit/f77630a9e971cae1f921292ea31d9d40a4b096b8

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'} Message: Import wiredtiger-wiredtiger-2.7.0-650-g5cdd3e3.tar.gz from wiredtiger branch mongodb-3.2 ref: 07966a4..5cdd3e3 SERVER-22437 Coverity analysis defect 77704: Redundant test SERVER-22438 Coverity analysis defect 77705: Dereference before null check SERVER-22676 WiredTiger fails to open databases created by 3.0.0 or 3.0.1 WT-2130 Improve on-disk page utlilization with random workloads WT-2215 WT_LSN needs to support atomic reads and updates WT-2295 WT_SESSION.create does a full-scan of the main table WT-2352 Allow build and test without requiring lz4 WT-2356 log scan advances to next log file on partially written record WT-2363 Remove built in support for bzip2 WT-2368 row-store can pass garbage keys to collator functions WT-2369 Use C compiler to detect headers instead of C++ compiler WT-2371 parent split cannot access the page after page-index swap WT-2372 WiredTiger windows builder fails with C4005 against the "inline" macro WT-2377 WTPERF doesn't compile in Windows under MSVC WT-2378 Tasks time out on LSM builder WT-2397 Cursor traversal from end of the tree skips records. WT-60 Big endian port Branch: v3.2 https://github.com/mongodb/mongo/commit/f77630a9e971cae1f921292ea31d9d40a4b096b8

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                1 year, 4 weeks ago
                Date of 1st Reply: