Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-1893

Stress test failure heap use after free by eviction

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: WT2.5.3
    • Fix Version/s: WT2.6.0
    • Labels:
      None
    • Environment:
      Jenkins
    • # Replies:
      1
    • Last comment by Customer:
      true

      Description

      There was a sanitizer stress test failure:

      ==6099==ERROR: AddressSanitizer: heap-use-after-free on address 0x61b00231ce9c at pc 0x5c99a5 bp 0x7f77a1ffd1f0 sp 0x7f77a1ffd1e8
      READ of size 4 at 0x61b00231ce9c thread T1
          #0 0x5c99a4 in __evict_walk /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/evict/evict_lru.c:967
          #1 0x5c5c7d in __evict_lru_walk /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/evict/evict_lru.c:790
          #2 0x5bea6a in __evict_pass /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/evict/evict_lru.c:534
          #3 0x5ac212 in __evict_server /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/evict/evict_lru.c:168
          #4 0x37ce207ee4 in start_thread (/lib64/libpthread.so.0+0x37ce207ee4)
          #5 0x37cdaf4d1c in __clone (/lib64/libc.so.6+0x37cdaf4d1c)
       
      0x61b00231ce9c is located 28 bytes inside of 1664-byte region [0x61b00231ce80,0x61b00231d500)
      freed by thread T0 here:
          #0 0x48e2a9 in free (/home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/t+0x48e2a9)
          #1 0x694f3f in __wt_free_int /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/os_posix/os_alloc.c:248
          #2 0xb3076f in __wt_conn_dhandle_discard_single /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/conn/conn_dhandle.c:772
          #3 0xb32c5e in __wt_conn_dhandle_discard /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/conn/conn_dhandle.c:805
          #4 0x53f457 in __wt_connection_close /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/conn/conn_open.c:121
          #5 0x4f793c in __conn_close /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/conn/conn_api.c:837
          #6 0x4db7f0 in wts_close /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/../../../test/format/wts.c:418
          #7 0x4cb6f2 in main /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/../../../test/format/t.c:223
          #8 0x37cda21d64 in __libc_start_main (/lib64/libc.so.6+0x37cda21d64)
      

      The eviction server is accessing a handle after it is free'd by connection close. A potential fix would be to stop the eviction server before closing all data handles.

        Issue Links

          Activity

          • Comments
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: Shut down the eviction server before closing file handles to avoid a race.

          refs WT-1893
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/c8fdd9c676c4a24bee6328a56cf7fd074cd045e0

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Shut down the eviction server before closing file handles to avoid a race. refs WT-1893 Branch: develop https://github.com/wiredtiger/wiredtiger/commit/c8fdd9c676c4a24bee6328a56cf7fd074cd045e0

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                2 years, 1 week, 6 days ago
                Date of 1st Reply: