Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2796

Memory leak in reconciliation uncovered by stress testing

    Details

    • # Replies:
      9
    • Last comment by Customer:
      true

      Description

      The Jenkins stress tests detected memory leaks. For example:

       
      Direct leak of 1060 byte(s) in 53 object(s) allocated from:
          #0 0x4a5ce0 in calloc (/mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/t+0x4a5ce0)
          #1 0x746fff in __wt_calloc /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/os_common/os_alloc.c:51:11
          #2 0xcb25eb in __wt_update_alloc /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/btree/row_modify.c:277:2
          #3 0xc716c6 in __wt_col_modify /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/btree/col_modify.c:151:4
          #4 0xff5265 in __cursor_col_modify /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/btree/bt_cursor.c:266:10
          #5 0xff9997 in __wt_btcur_remove /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/btree/bt_cursor.c:707:10
          #6 0xd85a7b in __curfile_remove /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/cursor/cur_file.c:331:2
          #7 0x4e5620 in col_remove /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/../../../test/format/ops.c:1392:9
          #8 0x4ddf28 in ops /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/../../../test/format/ops.c:611:11
          #9 0x7f049753adf2 in start_thread (/lib64/libpthread.so.0+0x7df2)
      

      The failure can be seen here http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-sanitizer/10560/console

      It is likely that this leak was introduced by the recent changes to scrub dirty pages. Doesn't reproduce quickly.

        Activity

        Hide
        alexander.gorrod Alexander Gorrod added a comment -

        The test/format configuration file that triggered the failure:

        ############################################
        #  RUN PARAMETERS
        ############################################
        abort=0
        auto_throttle=1
        backups=0
        bitcnt=4
        bloom=1
        bloom_bit_count=34
        bloom_hash_count=16
        bloom_oldest=0
        cache=9
        checkpoints=1
        checksum=on
        chunk_size=9
        compaction=0
        compression=lz4
        data_extend=0
        data_source=table
        delete_pct=28
        dictionary=0
        direct_io=0
        encryption=none
        evict_max=4
        file_type=variable-length column-store
        firstfit=0
        huffman_key=0
        huffman_value=0
        in_memory=0
        insert_pct=33
        internal_key_truncation=1
        internal_page_max=13
        isolation=read-uncommitted
        key_gap=12
        key_max=47
        key_min=22
        leaf_page_max=17
        leak_memory=0
        logging=0
        logging_archive=0
        logging_compression=none
        logging_prealloc=1
        long_running_txn=0
        lsm_worker_threads=3
        merge_max=17
        mmap=1
        ops=100000
        prefix_compression=1
        prefix_compression_min=3
        quiet=1
        repeat_data_pct=83
        reverse=0
        rows=100000
        runs=1
        rebalance=1
        salvage=1
        split_pct=57
        statistics=0
        statistics_server=0
        threads=9
        timer=20
        transaction-frequency=6
        value_max=2964
        value_min=5
        verify=1
        wiredtiger_config=
        write_pct=2
        ############################################
        

        Show
        alexander.gorrod Alexander Gorrod added a comment - The test/format configuration file that triggered the failure: ############################################ # RUN PARAMETERS ############################################ abort=0 auto_throttle=1 backups=0 bitcnt=4 bloom=1 bloom_bit_count=34 bloom_hash_count=16 bloom_oldest=0 cache=9 checkpoints=1 checksum=on chunk_size=9 compaction=0 compression=lz4 data_extend=0 data_source=table delete_pct=28 dictionary=0 direct_io=0 encryption=none evict_max=4 file_type=variable-length column-store firstfit=0 huffman_key=0 huffman_value=0 in_memory=0 insert_pct=33 internal_key_truncation=1 internal_page_max=13 isolation=read-uncommitted key_gap=12 key_max=47 key_min=22 leaf_page_max=17 leak_memory=0 logging=0 logging_archive=0 logging_compression=none logging_prealloc=1 long_running_txn=0 lsm_worker_threads=3 merge_max=17 mmap=1 ops=100000 prefix_compression=1 prefix_compression_min=3 quiet=1 repeat_data_pct=83 reverse=0 rows=100000 runs=1 rebalance=1 salvage=1 split_pct=57 statistics=0 statistics_server=0 threads=9 timer=20 transaction-frequency=6 value_max=2964 value_min=5 verify=1 wiredtiger_config= write_pct=2 ############################################
        Hide
        alexander.gorrod Alexander Gorrod added a comment -

        Somewhat interestingly there are a lot of memory leaks in the run. They seem to be one of two sizes, either 37 bytes or 2731 bytes. Most of the call stacks are of the form:

        Indirect leak of 10924 byte(s) in 4 object(s) allocated from:
            #0 0x4a5ce0 in calloc (/mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/t+0x4a5ce0)
            #1 0x746fff in __wt_calloc /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/os_common/os_alloc.c:51:11
            #2 0xcb25eb in __wt_update_alloc /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/btree/row_modify.c:277:2
            #3 0x84a242 in __rec_txn_read /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/reconcile/rec_write.c:1279:11
            #4 0x7fd6e9 in __rec_col_var /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/reconcile/rec_write.c:4388:5
            #5 0x7eaa23 in __wt_reconcile /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/reconcile/rec_write.c:406:9
            #6 0x654e83 in __evict_review /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/evict/evict_page.c:543:2
        

        Some are similar to the one posted in the ticket description.

        Show
        alexander.gorrod Alexander Gorrod added a comment - Somewhat interestingly there are a lot of memory leaks in the run. They seem to be one of two sizes, either 37 bytes or 2731 bytes. Most of the call stacks are of the form: Indirect leak of 10924 byte(s) in 4 object(s) allocated from: #0 0x4a5ce0 in calloc (/mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/test/format/t+0x4a5ce0) #1 0x746fff in __wt_calloc /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/os_common/os_alloc.c:51:11 #2 0xcb25eb in __wt_update_alloc /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/btree/row_modify.c:277:2 #3 0x84a242 in __rec_txn_read /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/reconcile/rec_write.c:1279:11 #4 0x7fd6e9 in __rec_col_var /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/reconcile/rec_write.c:4388:5 #5 0x7eaa23 in __wt_reconcile /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/reconcile/rec_write.c:406:9 #6 0x654e83 in __evict_review /mnt/fast/jenkins/workspace/wiredtiger-test-format-stress-sanitizer/build_posix/../src/evict/evict_page.c:543:2 Some are similar to the one posted in the ticket description.
        Hide
        alexander.gorrod Alexander Gorrod added a comment -

        I spent a bunch of time doing runs today. This appears to be specific to the WT-2764 branch which isn't a surprise given than the branch is doing a lot more scrubbing than develop.

        I've only been able to reproduce once all day, and that took a run of 20 minutes (i.e: the run is very slow). Not much useful information I'm afraid.

        Show
        alexander.gorrod Alexander Gorrod added a comment - I spent a bunch of time doing runs today. This appears to be specific to the WT-2764 branch which isn't a surprise given than the branch is doing a lot more scrubbing than develop. I've only been able to reproduce once all day, and that took a run of 20 minutes (i.e: the run is very slow). Not much useful information I'm afraid.
        Hide
        sue.loverso Sue LoVerso added a comment -

        Keith Bostic and I spent a long time on a call today. I have not had luck reproducing on my AWS box with clang yet (2+ hours).

        I'm going to add in some of his suggestions from slack into the ticket. Some suggestions on configuration changes to make it reproduce faster perhaps:

        • Try a bigger cache (it may hit faster, or it may make it go away) CONFIG:cache
        • Try more threads CONFIG:threads
        • Turn off salvage, verify and rebalance in CONFIG to make each iteration go faster.
        • Try smaller CONFIG:leaf_page_max.
        • Turn off overflow, make CONFIG:value_max smaller.
        • Try larger CONFIG:write_pct.
        • Try CONFIG:isolation=snapshot so that test/format does better testing of getting the correct key/value pairs. If we are allocating update structures but not correctly keeping or applying after reconciliation then snapshot may detect it at the time it happens instead of sanitizer detecting at the end of a run.

        First step is getting a reproducer. This is already not easy (for me). We could record the flags that were sent to wt_reconcile, in evict_review, on the page, so we know how it was called.

        Show
        sue.loverso Sue LoVerso added a comment - Keith Bostic and I spent a long time on a call today. I have not had luck reproducing on my AWS box with clang yet (2+ hours). I'm going to add in some of his suggestions from slack into the ticket. Some suggestions on configuration changes to make it reproduce faster perhaps: Try a bigger cache (it may hit faster, or it may make it go away) CONFIG:cache Try more threads CONFIG:threads Turn off salvage, verify and rebalance in CONFIG to make each iteration go faster. Try smaller CONFIG:leaf_page_max . Turn off overflow, make CONFIG:value_max smaller. Try larger CONFIG:write_pct . Try CONFIG:isolation=snapshot so that test/format does better testing of getting the correct key/value pairs. If we are allocating update structures but not correctly keeping or applying after reconciliation then snapshot may detect it at the time it happens instead of sanitizer detecting at the end of a run. First step is getting a reproducer. This is already not easy (for me). We could record the flags that were sent to wt_reconcile , in evict_review , on the page, so we know how it was called.
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2796 Fix a memory leak when using the lookaside table. (#2914)
        Branch: develop
        https://github.com/wiredtiger/wiredtiger/commit/d7a15d1798054c22fae98bb90b9badb24792d966

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2796 Fix a memory leak when using the lookaside table. (#2914) Branch: develop https://github.com/wiredtiger/wiredtiger/commit/d7a15d1798054c22fae98bb90b9badb24792d966
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2796 Fix a memory leak when using the lookaside table. (#2914)
        Branch: mongodb-3.4
        https://github.com/wiredtiger/wiredtiger/commit/d7a15d1798054c22fae98bb90b9badb24792d966

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2796 Fix a memory leak when using the lookaside table. (#2914) Branch: mongodb-3.4 https://github.com/wiredtiger/wiredtiger/commit/d7a15d1798054c22fae98bb90b9badb24792d966
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

        Message: Import wiredtiger-wiredtiger-2.8.0-589-ga9e9696.tar.gz from wiredtiger branch mongodb-3.4

        ref: d8fb874..a9e9696
        for: 3.3.11

        SERVER-24971 Excessive memory held by sessions when application threads do evictions
        WT-1162 Add latency to Jenkins wtperf tests and plots
        WT-2026 Maximum pages size at eviction too large
        WT-2239 Make sure LSM cursors read up to date dsk_gen, it was racing with compact
        WT-2353 Failure to create async threads as part of a wiredtiger_open call will cause a hang
        WT-2380 Make scripts fail if code doesn't match style
        WT-2486 Update make check so that it runs faster
        WT-2578 remove write barriers from the TAILQ_INSERT_XXX macros
        WT-2648 cache-line alignment for new ports
        WT-2665 Limit allocator fragmentation in WiredTiger
        WT-2693 Check open_cursor error paths for consistent handling
        WT-2708 split child-update race with reconciliation/eviction
        WT-2711 Change statistics log configuration options
        WT-2728 Don't re-read log file headers during log_flush
        WT-2729 Focus eviction walks in largest trees
        WT-2730 cursor next/prev can return the wrong key/value pair when crossing a page boundary
        WT-2731 Raw compression can create pages that are larger than expected
        WT-2732 Coverity analysis defect 99665: Redundant test
        WT-2737 Scrub dirty pages rather than evicting them
        WT-2738 Remove the ability to change the default checkpoint name
        WT-2739 pluggable file systems documentation cleanups
        WT-2743 Thread count statistics always report 0
        WT-2744 partial line even with line buffering set
        WT-2746 track checkpoint I/O separately from eviction I/O
        WT-2751 column-store statistics incorrectly calculates the number of entries
        WT-2752 Fixes to zipfian wtperf workload config
        WT-2755 flexelint configuration treats size_t as 4B type
        WT-2756 Upgrade the autoconf archive package to check for swig 3.0
        WT-2757 Column tables behave differently when column names are provided
        WT-2759 Releasing the hot-backup lock doesn't require the schema lock.
        WT-2760 Fix a bug in backup related to directory sync. Change the filesystem API to make durable the default
        WT-2762 wtstats tool fails if checkpoint runs
        WT-2763 Unit test test_intpack failing on OSX
        WT-2764 Optimize checkpoints to reduce throughput disruption
        WT-2765 wt dump: indices need to be shown in the dump output
        WT-2767 test suite needs way to run an individual scenario
        WT-2769 Update documentation to reflect correct limits of memory_page_max
        WT-2770 Add statistics tracking schema operations
        WT-2772 Investigate log performance testing weirdness
        WT-2773 search_near in indexes does not find exact matches
        WT-2774 minor cleanups/improvements
        WT-2778 Python test suite: make scenario initialization consistent
        WT-2779 Raw compression created unexpectedly large pages on disk
        WT-2781 Enhance bulk cursor option with an option to return immediately on contention
        WT-2782 Missing a fs_directory_list_free in ex_file_system.c
        WT-2785 Scrub dirty pages rather than evicting them: single-page reconciliation
        WT-2791 Enhance OS X Evergreen unit test
        WT-2793 wtperf config improvements
        WT-2796 Memory leak in reconciliation uncovered by stress testing
        WT-2798 Crash vulnerability with nojournal after create during checkpoint
        WT-2800 Illegal file format in test/format on PPC
        WT-2801 Crash vulnerability from eviction of metadata during checkpoint
        WT-2802 Transaction commit causes heap-use-after free
        WT-2803 Add verbose functionality to WT Evergreen tests
        WT-2804 Don't read values in a tree without a snapshot
        WT-2805 Infinite recursion if error streams fail
        WT-2806 wtperf allocation size off-by-one
        Branch: master
        https://github.com/mongodb/mongo/commit/c4612847ed88c5e0500f0e9ecb2ecdcb49621cf1

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'} Message: Import wiredtiger-wiredtiger-2.8.0-589-ga9e9696.tar.gz from wiredtiger branch mongodb-3.4 ref: d8fb874..a9e9696 for: 3.3.11 SERVER-24971 Excessive memory held by sessions when application threads do evictions WT-1162 Add latency to Jenkins wtperf tests and plots WT-2026 Maximum pages size at eviction too large WT-2239 Make sure LSM cursors read up to date dsk_gen, it was racing with compact WT-2353 Failure to create async threads as part of a wiredtiger_open call will cause a hang WT-2380 Make scripts fail if code doesn't match style WT-2486 Update make check so that it runs faster WT-2578 remove write barriers from the TAILQ_INSERT_XXX macros WT-2648 cache-line alignment for new ports WT-2665 Limit allocator fragmentation in WiredTiger WT-2693 Check open_cursor error paths for consistent handling WT-2708 split child-update race with reconciliation/eviction WT-2711 Change statistics log configuration options WT-2728 Don't re-read log file headers during log_flush WT-2729 Focus eviction walks in largest trees WT-2730 cursor next/prev can return the wrong key/value pair when crossing a page boundary WT-2731 Raw compression can create pages that are larger than expected WT-2732 Coverity analysis defect 99665: Redundant test WT-2737 Scrub dirty pages rather than evicting them WT-2738 Remove the ability to change the default checkpoint name WT-2739 pluggable file systems documentation cleanups WT-2743 Thread count statistics always report 0 WT-2744 partial line even with line buffering set WT-2746 track checkpoint I/O separately from eviction I/O WT-2751 column-store statistics incorrectly calculates the number of entries WT-2752 Fixes to zipfian wtperf workload config WT-2755 flexelint configuration treats size_t as 4B type WT-2756 Upgrade the autoconf archive package to check for swig 3.0 WT-2757 Column tables behave differently when column names are provided WT-2759 Releasing the hot-backup lock doesn't require the schema lock. WT-2760 Fix a bug in backup related to directory sync. Change the filesystem API to make durable the default WT-2762 wtstats tool fails if checkpoint runs WT-2763 Unit test test_intpack failing on OSX WT-2764 Optimize checkpoints to reduce throughput disruption WT-2765 wt dump: indices need to be shown in the dump output WT-2767 test suite needs way to run an individual scenario WT-2769 Update documentation to reflect correct limits of memory_page_max WT-2770 Add statistics tracking schema operations WT-2772 Investigate log performance testing weirdness WT-2773 search_near in indexes does not find exact matches WT-2774 minor cleanups/improvements WT-2778 Python test suite: make scenario initialization consistent WT-2779 Raw compression created unexpectedly large pages on disk WT-2781 Enhance bulk cursor option with an option to return immediately on contention WT-2782 Missing a fs_directory_list_free in ex_file_system.c WT-2785 Scrub dirty pages rather than evicting them: single-page reconciliation WT-2791 Enhance OS X Evergreen unit test WT-2793 wtperf config improvements WT-2796 Memory leak in reconciliation uncovered by stress testing WT-2798 Crash vulnerability with nojournal after create during checkpoint WT-2800 Illegal file format in test/format on PPC WT-2801 Crash vulnerability from eviction of metadata during checkpoint WT-2802 Transaction commit causes heap-use-after free WT-2803 Add verbose functionality to WT Evergreen tests WT-2804 Don't read values in a tree without a snapshot WT-2805 Infinite recursion if error streams fail WT-2806 wtperf allocation size off-by-one Branch: master https://github.com/mongodb/mongo/commit/c4612847ed88c5e0500f0e9ecb2ecdcb49621cf1
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2796 Fix a memory leak when using the lookaside table. (#2914)
        Branch: mongodb-3.2
        https://github.com/wiredtiger/wiredtiger/commit/d7a15d1798054c22fae98bb90b9badb24792d966

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2796 Fix a memory leak when using the lookaside table. (#2914) Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/d7a15d1798054c22fae98bb90b9badb24792d966
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

        Message: Import wiredtiger: 911c940adab547d36ac305fc627a79e637fa3c40 from branch mongodb-3.2

        ref: dddca65..911c940ada
        for: 3.2.10

        SERVER-24971 Excessive memory held by sessions when application threads do evictions
        SERVER-25843 Coverity analysis defect 99856: Redundant test
        SERVER-25845 Coverity analysis defect 99859: Explicit null dereferenced
        SERVER-25846 Coverity analysis defect 99861: Dereference after null check
        WT-1162 Add latency to Jenkins wtperf tests and plots
        WT-2026 Maximum pages size at eviction too large
        WT-2221 Document which statistics are available via a "fast" configuration vs. an "all" configuration
        WT-2233 Investigate changing when the eviction server switches to aggressive mode.
        WT-2239 Make sure LSM cursors read up to date dsk_gen, it was racing with compact
        WT-2323 Allocate a transaction id at the beginning of join cursor iteration
        WT-2353 Failure to create async threads as part of a wiredtiger_open call will cause a hang
        WT-2380 Make scripts fail if code doesn't match style
        WT-2486 Update make check so that it runs faster
        WT-2555 make format run on Windows
        WT-2578 remove write barriers from the TAILQ_INSERT_XXX macros
        WT-2631 nullptr is passed for parameters marked with attribute non-null
        WT-2638 ftruncate may not be supported
        WT-2645 wt dump: push the complexity of collecting metadata into a dump cursor
        WT-2648 cache-line alignment for new ports
        WT-2665 Limit allocator fragmentation in WiredTiger
        WT-2678 The metadata should not imply that an empty value is true
        WT-2688 configure --enable-python doesn't check for availability of swig
        WT-2693 Check open_cursor error paths for consistent handling
        WT-2695 Integrate s390x accelerated crc32c support
        WT-2708 split child-update race with reconciliation/eviction
        WT-2711 Change statistics log configuration options
        WT-2719 add fuzz testing for WiredTiger options and reconfiguration.
        WT-2728 Don't re-read log file headers during log_flush
        WT-2729 Focus eviction walks in largest trees
        WT-2730 cursor next/prev can return the wrong key/value pair when crossing a page boundary
        WT-2731 Raw compression can create pages that are larger than expected
        WT-2732 Coverity analysis defect 99665: Redundant test
        WT-2734 Improve documentation of eviction behavior
        WT-2737 Scrub dirty pages rather than evicting them
        WT-2738 Remove the ability to change the default checkpoint name
        WT-2739 pluggable file systems documentation cleanups
        WT-2743 Thread count statistics always report 0
        WT-2744 partial line even with line buffering set
        WT-2746 track checkpoint I/O separately from eviction I/O
        WT-2751 column-store statistics incorrectly calculates the number of entries
        WT-2752 Fixes to zipfian wtperf workload config
        WT-2755 flexelint configuration treats size_t as 4B type
        WT-2756 Upgrade the autoconf archive package to check for swig 3.0
        WT-2757 Column tables behave differently when column names are provided
        WT-2759 Releasing the hot-backup lock doesn't require the schema lock.
        WT-2760 Fix a bug in backup related to directory sync. Change the filesystem API to make durable the default
        WT-2762 wtstats tool fails if checkpoint runs
        WT-2763 Unit test test_intpack failing on OSX
        WT-2764 Optimize checkpoints to reduce throughput disruption
        WT-2765 wt dump: indices need to be shown in the dump output
        WT-2766 Don't count eviction of lookaside file pages for the purpose of checking stuck cache
        WT-2767 test suite needs way to run an individual scenario
        WT-2769 Update documentation to reflect correct limits of memory_page_max
        WT-2770 Add statistics tracking schema operations
        WT-2772 Investigate log performance testing weirdness
        WT-2773 search_near in indexes does not find exact matches
        WT-2774 minor cleanups/improvements
        WT-2778 Python test suite: make scenario initialization consistent
        WT-2779 Raw compression created unexpectedly large pages on disk
        WT-2781 Enhance bulk cursor option with an option to return immediately on contention
        WT-2782 Missing a fs_directory_list_free in ex_file_system.c
        WT-2783 wtperf multi-btree.wtperf dumps core on Mac
        WT-2785 Scrub dirty pages rather than evicting them: single-page reconciliation
        WT-2787 Include src/include/wiredtiger_ext.h is problematic
        WT-2788 Java: freed memory overwrite during handle close can cause JNI crash
        WT-2791 Enhance OS X Evergreen unit test
        WT-2793 wtperf config improvements
        WT-2795 Update documentation around read-only configuration
        WT-2796 Memory leak in reconciliation uncovered by stress testing
        WT-2798 Crash vulnerability with nojournal after create during checkpoint
        WT-2800 Illegal file format in test/format on PPC
        WT-2801 Crash vulnerability from eviction of metadata during checkpoint
        WT-2802 Transaction commit causes heap-use-after free
        WT-2803 Add verbose functionality to WT Evergreen tests
        WT-2804 Don't read values in a tree without a snapshot
        WT-2805 Infinite recursion if error streams fail
        WT-2806 wtperf allocation size off-by-one
        WT-2807 Switch Jenkins performance tests to tcmalloc
        WT-2811 Reconciliation asserts that transaction time has gone backwards
        WT-2812 Error when reconfiguring cache targets
        WT-2813 small cache usage stuck even with large cache
        WT-2814 Enhance wtperf to support single-op truncate mode
        WT-2816 Improve WiredTiger eviction performance
        WT-2817 Investigate performance regression in develop, add workload to wtperf/runners
        WT-2818 The page visibility check when queuing pages for eviction is overly restrictive
        WT-2820 add gcc warn_unused_result attribute
        WT-2822 panic mutex and other functions that cannot fail
        WT-2823 support file handles without a truncate method
        WT-2824 wtperf displays connection and table create configurations twice
        WT-2826 clang38 false positive on uninitialized variable.
        WT-2827 checkpoint log_size configuration improvements
        WT-2828 Make long wtperf tests reflect mongoDB usage
        WT-2829 Switch automated testing to use enable-strict configure option
        WT-2832 Python test uses hard-coded temporary directory
        WT-2834 Join cursor: discrepancy with bloom filters
        WT-2835 WT_CONNECTION.leak-memory can skip memory map and cache cleanup
        WT-2838 Don't free session handles on close if leak memory is configured
        WT-2839 lint: Ignoring return value of function
        WT-2840 clang analysis: garbage values
        WT-2841 Jenkins Valgrind runner is reporting errors in test wt2719_reconfig
        WT-2842 split wtperf's configuration into per-database and per-run parts
        WT-2843 Fix a bug in recovery if there is no filesystem truncate support
        WT-2846 Several bugs related to reconfiguring eviction server at runtime
        WT-2847 Merge fair locks into read/write locks.
        WT-2850 clang 4.1 attribute warnings when building
        WT-2853 Multi threaded reader writer example shows temporary slowdown or lockup
        WT-2857 POSIX ftruncate calls should be #ifdef'd HAVE_FTRUNCATE
        WT-2862 Fix lint error in test case for forced eviction with multiple cursors
        WT-2863 Support UTF-8 paths on Windows
        WT-2865 eviction thread error failure
        WT-2866 Eviction server algorithm tuning
        WT-2867 Review and fix barrier usage in __lsm_tree_close
        WT-2868 Add sample_interval to checkpoint-stress wtperf config
        WT-2869 Performance regression on secondaries
        WT-2870 Rename wtperf checkpoint schema jobs
        WT-2871 __wt_verbose has the wrong GCC format attributes
        WT-2872 Recent stuck cache test/stress failures.
        WT-2873 Refactor CRC32 code
        WT-2875 Test test_wt2853_perf can run too long under valgrind
        WT-2876 Extend wtperf to support a log like table
        WT-2878 Verbose changes affected performance
        WT-2881 Add -Wpedantic to clang compiler warning flags
        WT-2883 wiredtiger_open with verbose=handleops recursive loop
        WT-2885 __wt_checkpoint_signal lint
        WT-2886 Decide how in-memory configuration and eviction_dirty_target interact
        WT-2888 Switch functions to return void where possible
        WT-2892 hot backup can race with block truncate
        WT-2896 Coverity #1362535: resource leak
        WT-2897 Checkpoints can become corrupted on failure
        WT-2901 Add option to disable checkpoint dirty stepdown phase
        WT-2903 Reduce the impact of checkpoint scrubbing on applications
        Branch: v3.2
        https://github.com/mongodb/mongo/commit/7d2acd6395ec84beca34718a75371bc11f0c9f60

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'} Message: Import wiredtiger: 911c940adab547d36ac305fc627a79e637fa3c40 from branch mongodb-3.2 ref: dddca65..911c940ada for: 3.2.10 SERVER-24971 Excessive memory held by sessions when application threads do evictions SERVER-25843 Coverity analysis defect 99856: Redundant test SERVER-25845 Coverity analysis defect 99859: Explicit null dereferenced SERVER-25846 Coverity analysis defect 99861: Dereference after null check WT-1162 Add latency to Jenkins wtperf tests and plots WT-2026 Maximum pages size at eviction too large WT-2221 Document which statistics are available via a "fast" configuration vs. an "all" configuration WT-2233 Investigate changing when the eviction server switches to aggressive mode. WT-2239 Make sure LSM cursors read up to date dsk_gen, it was racing with compact WT-2323 Allocate a transaction id at the beginning of join cursor iteration WT-2353 Failure to create async threads as part of a wiredtiger_open call will cause a hang WT-2380 Make scripts fail if code doesn't match style WT-2486 Update make check so that it runs faster WT-2555 make format run on Windows WT-2578 remove write barriers from the TAILQ_INSERT_XXX macros WT-2631 nullptr is passed for parameters marked with attribute non-null WT-2638 ftruncate may not be supported WT-2645 wt dump: push the complexity of collecting metadata into a dump cursor WT-2648 cache-line alignment for new ports WT-2665 Limit allocator fragmentation in WiredTiger WT-2678 The metadata should not imply that an empty value is true WT-2688 configure --enable-python doesn't check for availability of swig WT-2693 Check open_cursor error paths for consistent handling WT-2695 Integrate s390x accelerated crc32c support WT-2708 split child-update race with reconciliation/eviction WT-2711 Change statistics log configuration options WT-2719 add fuzz testing for WiredTiger options and reconfiguration. WT-2728 Don't re-read log file headers during log_flush WT-2729 Focus eviction walks in largest trees WT-2730 cursor next/prev can return the wrong key/value pair when crossing a page boundary WT-2731 Raw compression can create pages that are larger than expected WT-2732 Coverity analysis defect 99665: Redundant test WT-2734 Improve documentation of eviction behavior WT-2737 Scrub dirty pages rather than evicting them WT-2738 Remove the ability to change the default checkpoint name WT-2739 pluggable file systems documentation cleanups WT-2743 Thread count statistics always report 0 WT-2744 partial line even with line buffering set WT-2746 track checkpoint I/O separately from eviction I/O WT-2751 column-store statistics incorrectly calculates the number of entries WT-2752 Fixes to zipfian wtperf workload config WT-2755 flexelint configuration treats size_t as 4B type WT-2756 Upgrade the autoconf archive package to check for swig 3.0 WT-2757 Column tables behave differently when column names are provided WT-2759 Releasing the hot-backup lock doesn't require the schema lock. WT-2760 Fix a bug in backup related to directory sync. Change the filesystem API to make durable the default WT-2762 wtstats tool fails if checkpoint runs WT-2763 Unit test test_intpack failing on OSX WT-2764 Optimize checkpoints to reduce throughput disruption WT-2765 wt dump: indices need to be shown in the dump output WT-2766 Don't count eviction of lookaside file pages for the purpose of checking stuck cache WT-2767 test suite needs way to run an individual scenario WT-2769 Update documentation to reflect correct limits of memory_page_max WT-2770 Add statistics tracking schema operations WT-2772 Investigate log performance testing weirdness WT-2773 search_near in indexes does not find exact matches WT-2774 minor cleanups/improvements WT-2778 Python test suite: make scenario initialization consistent WT-2779 Raw compression created unexpectedly large pages on disk WT-2781 Enhance bulk cursor option with an option to return immediately on contention WT-2782 Missing a fs_directory_list_free in ex_file_system.c WT-2783 wtperf multi-btree.wtperf dumps core on Mac WT-2785 Scrub dirty pages rather than evicting them: single-page reconciliation WT-2787 Include src/include/wiredtiger_ext.h is problematic WT-2788 Java: freed memory overwrite during handle close can cause JNI crash WT-2791 Enhance OS X Evergreen unit test WT-2793 wtperf config improvements WT-2795 Update documentation around read-only configuration WT-2796 Memory leak in reconciliation uncovered by stress testing WT-2798 Crash vulnerability with nojournal after create during checkpoint WT-2800 Illegal file format in test/format on PPC WT-2801 Crash vulnerability from eviction of metadata during checkpoint WT-2802 Transaction commit causes heap-use-after free WT-2803 Add verbose functionality to WT Evergreen tests WT-2804 Don't read values in a tree without a snapshot WT-2805 Infinite recursion if error streams fail WT-2806 wtperf allocation size off-by-one WT-2807 Switch Jenkins performance tests to tcmalloc WT-2811 Reconciliation asserts that transaction time has gone backwards WT-2812 Error when reconfiguring cache targets WT-2813 small cache usage stuck even with large cache WT-2814 Enhance wtperf to support single-op truncate mode WT-2816 Improve WiredTiger eviction performance WT-2817 Investigate performance regression in develop, add workload to wtperf/runners WT-2818 The page visibility check when queuing pages for eviction is overly restrictive WT-2820 add gcc warn_unused_result attribute WT-2822 panic mutex and other functions that cannot fail WT-2823 support file handles without a truncate method WT-2824 wtperf displays connection and table create configurations twice WT-2826 clang38 false positive on uninitialized variable. WT-2827 checkpoint log_size configuration improvements WT-2828 Make long wtperf tests reflect mongoDB usage WT-2829 Switch automated testing to use enable-strict configure option WT-2832 Python test uses hard-coded temporary directory WT-2834 Join cursor: discrepancy with bloom filters WT-2835 WT_CONNECTION.leak-memory can skip memory map and cache cleanup WT-2838 Don't free session handles on close if leak memory is configured WT-2839 lint: Ignoring return value of function WT-2840 clang analysis: garbage values WT-2841 Jenkins Valgrind runner is reporting errors in test wt2719_reconfig WT-2842 split wtperf's configuration into per-database and per-run parts WT-2843 Fix a bug in recovery if there is no filesystem truncate support WT-2846 Several bugs related to reconfiguring eviction server at runtime WT-2847 Merge fair locks into read/write locks. WT-2850 clang 4.1 attribute warnings when building WT-2853 Multi threaded reader writer example shows temporary slowdown or lockup WT-2857 POSIX ftruncate calls should be #ifdef'd HAVE_FTRUNCATE WT-2862 Fix lint error in test case for forced eviction with multiple cursors WT-2863 Support UTF-8 paths on Windows WT-2865 eviction thread error failure WT-2866 Eviction server algorithm tuning WT-2867 Review and fix barrier usage in __lsm_tree_close WT-2868 Add sample_interval to checkpoint-stress wtperf config WT-2869 Performance regression on secondaries WT-2870 Rename wtperf checkpoint schema jobs WT-2871 __wt_verbose has the wrong GCC format attributes WT-2872 Recent stuck cache test/stress failures. WT-2873 Refactor CRC32 code WT-2875 Test test_wt2853_perf can run too long under valgrind WT-2876 Extend wtperf to support a log like table WT-2878 Verbose changes affected performance WT-2881 Add -Wpedantic to clang compiler warning flags WT-2883 wiredtiger_open with verbose=handleops recursive loop WT-2885 __wt_checkpoint_signal lint WT-2886 Decide how in-memory configuration and eviction_dirty_target interact WT-2888 Switch functions to return void where possible WT-2892 hot backup can race with block truncate WT-2896 Coverity #1362535: resource leak WT-2897 Checkpoints can become corrupted on failure WT-2901 Add option to disable checkpoint dirty stepdown phase WT-2903 Reduce the impact of checkpoint scrubbing on applications Branch: v3.2 https://github.com/mongodb/mongo/commit/7d2acd6395ec84beca34718a75371bc11f0c9f60

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Days since reply:
              40 weeks, 5 days ago
              Date of 1st Reply: