Author:

Message: WT-2804 Don't read values in a tree without a snapshot. (#2924)

• Improve two recent assertions, one from WT-2798 relating to writing metadata updates to disk that are part of a running transaction, and another from WT-2802 that checks that we don't try to copy values from a cursor without a transaction pinned. The latter doesn't apply to cursors on checkpoints (including chunk cursors in an LSM tree).

• Copy cursor values before rollback in autocommit.

If an autocommit operation such as WT_CURSOR::update touches multiple trees (e.g., multiple column groups in a table, or index updates, or multiple chunks in an LSM tree), then some cursors may have consumed the application's key/value pair when the operation has to roll back. Take a copy of any such values before attempting to retry the operation.
Branch: mongodb-3.4
https://github.com/wiredtiger/wiredtiger/commit/41eb2dcaac1ff25654d1503f5e29714576ff8d81

Author:

Message: Import wiredtiger-wiredtiger-2.8.0-589-ga9e9696.tar.gz from wiredtiger branch mongodb-3.4

ref: d8fb874..a9e9696
for: 3.3.11

SERVER-24971 Excessive memory held by sessions when application threads do evictions
WT-1162 Add latency to Jenkins wtperf tests and plots
WT-2026 Maximum pages size at eviction too large
WT-2239 Make sure LSM cursors read up to date dsk_gen, it was racing with compact
WT-2353 Failure to create async threads as part of a wiredtiger_open call will cause a hang
WT-2380 Make scripts fail if code doesn't match style
WT-2486 Update make check so that it runs faster
WT-2578 remove write barriers from the TAILQ_INSERT_XXX macros
WT-2648 cache-line alignment for new ports
WT-2665 Limit allocator fragmentation in WiredTiger
WT-2693 Check open_cursor error paths for consistent handling
WT-2708 split child-update race with reconciliation/eviction
WT-2711 Change statistics log configuration options
WT-2728 Don't re-read log file headers during log_flush
WT-2729 Focus eviction walks in largest trees
WT-2730 cursor next/prev can return the wrong key/value pair when crossing a page boundary
WT-2731 Raw compression can create pages that are larger than expected
WT-2732 Coverity analysis defect 99665: Redundant test
WT-2737 Scrub dirty pages rather than evicting them
WT-2738 Remove the ability to change the default checkpoint name
WT-2739 pluggable file systems documentation cleanups
WT-2743 Thread count statistics always report 0
WT-2744 partial line even with line buffering set
WT-2746 track checkpoint I/O separately from eviction I/O
WT-2751 column-store statistics incorrectly calculates the number of entries
WT-2752 Fixes to zipfian wtperf workload config
WT-2755 flexelint configuration treats size_t as 4B type
WT-2756 Upgrade the autoconf archive package to check for swig 3.0
WT-2757 Column tables behave differently when column names are provided
WT-2759 Releasing the hot-backup lock doesn't require the schema lock.
WT-2760 Fix a bug in backup related to directory sync. Change the filesystem API to make durable the default
WT-2762 wtstats tool fails if checkpoint runs
WT-2763 Unit test test_intpack failing on OSX
WT-2764 Optimize checkpoints to reduce throughput disruption
WT-2765 wt dump: indices need to be shown in the dump output
WT-2767 test suite needs way to run an individual scenario
WT-2769 Update documentation to reflect correct limits of memory_page_max
WT-2770 Add statistics tracking schema operations
WT-2772 Investigate log performance testing weirdness
WT-2773 search_near in indexes does not find exact matches
WT-2774 minor cleanups/improvements
WT-2778 Python test suite: make scenario initialization consistent
WT-2779 Raw compression created unexpectedly large pages on disk
WT-2781 Enhance bulk cursor option with an option to return immediately on contention
WT-2782 Missing a fs_directory_list_free in ex_file_system.c
WT-2785 Scrub dirty pages rather than evicting them: single-page reconciliation
WT-2791 Enhance OS X Evergreen unit test
WT-2793 wtperf config improvements
WT-2796 Memory leak in reconciliation uncovered by stress testing
WT-2798 Crash vulnerability with nojournal after create during checkpoint
WT-2800 Illegal file format in test/format on PPC
WT-2801 Crash vulnerability from eviction of metadata during checkpoint
WT-2802 Transaction commit causes heap-use-after free
WT-2803 Add verbose functionality to WT Evergreen tests
WT-2804 Don't read values in a tree without a snapshot
WT-2805 Infinite recursion if error streams fail
WT-2806 wtperf allocation size off-by-one
Branch: master
https://github.com/mongodb/mongo/commit/c4612847ed88c5e0500f0e9ecb2ecdcb49621cf1

Author:

Message: WT-2798 Fix data consistency bug with table creates during a checkpoint. (#2910)

When logging is disabled, a create operation (and potentially other
metadata updates) could write partially completed checkpoint metadata,
leaving on-disk files inconsistent until the checkpoint completes.
Branch: mongodb-3.2
https://github.com/wiredtiger/wiredtiger/commit/7e1a47dd45735a8ee98aaf24f60406a0bb682359

Author:

Message: WT-2804 Don't read values in a tree without a snapshot. (#2924)

• Improve two recent assertions, one from WT-2798 relating to writing metadata updates to disk that are part of a running transaction, and another from WT-2802 that checks that we don't try to copy values from a cursor without a transaction pinned. The latter doesn't apply to cursors on checkpoints (including chunk cursors in an LSM tree).

• Copy cursor values before rollback in autocommit.

If an autocommit operation such as WT_CURSOR::update touches multiple trees (e.g., multiple column groups in a table, or index updates, or multiple chunks in an LSM tree), then some cursors may have consumed the application's key/value pair when the operation has to roll back. Take a copy of any such values before attempting to retry the operation.
Branch: mongodb-3.2
https://github.com/wiredtiger/wiredtiger/commit/41eb2dcaac1ff25654d1503f5e29714576ff8d81

Author:

Message: Import wiredtiger: 911c940adab547d36ac305fc627a79e637fa3c40 from branch mongodb-3.2

ref: dddca65..911c940ada
for: 3.2.10

SERVER-24971 Excessive memory held by sessions when application threads do evictions
SERVER-25843 Coverity analysis defect 99856: Redundant test
SERVER-25845 Coverity analysis defect 99859: Explicit null dereferenced
SERVER-25846 Coverity analysis defect 99861: Dereference after null check
WT-1162 Add latency to Jenkins wtperf tests and plots
WT-2026 Maximum pages size at eviction too large
WT-2221 Document which statistics are available via a "fast" configuration vs. an "all" configuration
WT-2233 Investigate changing when the eviction server switches to aggressive mode.
WT-2239 Make sure LSM cursors read up to date dsk_gen, it was racing with compact
WT-2323 Allocate a transaction id at the beginning of join cursor iteration
WT-2353 Failure to create async threads as part of a wiredtiger_open call will cause a hang
WT-2380 Make scripts fail if code doesn't match style
WT-2486 Update make check so that it runs faster
WT-2555 make format run on Windows
WT-2578 remove write barriers from the TAILQ_INSERT_XXX macros
WT-2631 nullptr is passed for parameters marked with attribute non-null
WT-2638 ftruncate may not be supported
WT-2645 wt dump: push the complexity of collecting metadata into a dump cursor
WT-2648 cache-line alignment for new ports
WT-2665 Limit allocator fragmentation in WiredTiger
WT-2678 The metadata should not imply that an empty value is true
WT-2688 configure --enable-python doesn't check for availability of swig
WT-2693 Check open_cursor error paths for consistent handling
WT-2695 Integrate s390x accelerated crc32c support
WT-2708 split child-update race with reconciliation/eviction
WT-2711 Change statistics log configuration options
WT-2719 add fuzz testing for WiredTiger options and reconfiguration.
WT-2728 Don't re-read log file headers during log_flush
WT-2729 Focus eviction walks in largest trees
WT-2730 cursor next/prev can return the wrong key/value pair when crossing a page boundary
WT-2731 Raw compression can create pages that are larger than expected
WT-2732 Coverity analysis defect 99665: Redundant test
WT-2734 Improve documentation of eviction behavior
WT-2737 Scrub dirty pages rather than evicting them
WT-2738 Remove the ability to change the default checkpoint name
WT-2739 pluggable file systems documentation cleanups
WT-2743 Thread count statistics always report 0
WT-2744 partial line even with line buffering set
WT-2746 track checkpoint I/O separately from eviction I/O
WT-2751 column-store statistics incorrectly calculates the number of entries
WT-2752 Fixes to zipfian wtperf workload config
WT-2755 flexelint configuration treats size_t as 4B type
WT-2756 Upgrade the autoconf archive package to check for swig 3.0
WT-2757 Column tables behave differently when column names are provided
WT-2759 Releasing the hot-backup lock doesn't require the schema lock.
WT-2760 Fix a bug in backup related to directory sync. Change the filesystem API to make durable the default
WT-2762 wtstats tool fails if checkpoint runs
WT-2763 Unit test test_intpack failing on OSX
WT-2764 Optimize checkpoints to reduce throughput disruption
WT-2765 wt dump: indices need to be shown in the dump output
WT-2766 Don't count eviction of lookaside file pages for the purpose of checking stuck cache
WT-2767 test suite needs way to run an individual scenario
WT-2769 Update documentation to reflect correct limits of memory_page_max
WT-2770 Add statistics tracking schema operations
WT-2772 Investigate log performance testing weirdness
WT-2773 search_near in indexes does not find exact matches
WT-2774 minor cleanups/improvements
WT-2778 Python test suite: make scenario initialization consistent
WT-2779 Raw compression created unexpectedly large pages on disk
WT-2781 Enhance bulk cursor option with an option to return immediately on contention
WT-2782 Missing a fs_directory_list_free in ex_file_system.c
WT-2783 wtperf multi-btree.wtperf dumps core on Mac
WT-2785 Scrub dirty pages rather than evicting them: single-page reconciliation
WT-2787 Include src/include/wiredtiger_ext.h is problematic
WT-2788 Java: freed memory overwrite during handle close can cause JNI crash
WT-2791 Enhance OS X Evergreen unit test
WT-2793 wtperf config improvements
WT-2795 Update documentation around read-only configuration
WT-2796 Memory leak in reconciliation uncovered by stress testing
WT-2798 Crash vulnerability with nojournal after create during checkpoint
WT-2800 Illegal file format in test/format on PPC
WT-2801 Crash vulnerability from eviction of metadata during checkpoint
WT-2802 Transaction commit causes heap-use-after free
WT-2803 Add verbose functionality to WT Evergreen tests
WT-2804 Don't read values in a tree without a snapshot
WT-2805 Infinite recursion if error streams fail
WT-2806 wtperf allocation size off-by-one
WT-2807 Switch Jenkins performance tests to tcmalloc
WT-2811 Reconciliation asserts that transaction time has gone backwards
WT-2812 Error when reconfiguring cache targets
WT-2813 small cache usage stuck even with large cache
WT-2814 Enhance wtperf to support single-op truncate mode
WT-2816 Improve WiredTiger eviction performance
WT-2817 Investigate performance regression in develop, add workload to wtperf/runners
WT-2818 The page visibility check when queuing pages for eviction is overly restrictive
WT-2820 add gcc warn_unused_result attribute
WT-2822 panic mutex and other functions that cannot fail
WT-2823 support file handles without a truncate method
WT-2824 wtperf displays connection and table create configurations twice
WT-2826 clang38 false positive on uninitialized variable.
WT-2827 checkpoint log_size configuration improvements
WT-2828 Make long wtperf tests reflect mongoDB usage
WT-2829 Switch automated testing to use enable-strict configure option
WT-2832 Python test uses hard-coded temporary directory
WT-2834 Join cursor: discrepancy with bloom filters
WT-2835 WT_CONNECTION.leak-memory can skip memory map and cache cleanup
WT-2838 Don't free session handles on close if leak memory is configured
WT-2839 lint: Ignoring return value of function
WT-2840 clang analysis: garbage values
WT-2841 Jenkins Valgrind runner is reporting errors in test wt2719_reconfig
WT-2842 split wtperf's configuration into per-database and per-run parts
WT-2843 Fix a bug in recovery if there is no filesystem truncate support
WT-2846 Several bugs related to reconfiguring eviction server at runtime
WT-2847 Merge fair locks into read/write locks.
WT-2850 clang 4.1 attribute warnings when building
WT-2853 Multi threaded reader writer example shows temporary slowdown or lockup
WT-2857 POSIX ftruncate calls should be #ifdef'd HAVE_FTRUNCATE
WT-2862 Fix lint error in test case for forced eviction with multiple cursors
WT-2863 Support UTF-8 paths on Windows
WT-2865 eviction thread error failure
WT-2866 Eviction server algorithm tuning
WT-2867 Review and fix barrier usage in __lsm_tree_close
WT-2868 Add sample_interval to checkpoint-stress wtperf config
WT-2869 Performance regression on secondaries
WT-2870 Rename wtperf checkpoint schema jobs
WT-2871 __wt_verbose has the wrong GCC format attributes
WT-2872 Recent stuck cache test/stress failures.
WT-2873 Refactor CRC32 code
WT-2875 Test test_wt2853_perf can run too long under valgrind
WT-2876 Extend wtperf to support a log like table
WT-2878 Verbose changes affected performance
WT-2881 Add -Wpedantic to clang compiler warning flags
WT-2883 wiredtiger_open with verbose=handleops recursive loop
WT-2885 __wt_checkpoint_signal lint
WT-2886 Decide how in-memory configuration and eviction_dirty_target interact
WT-2888 Switch functions to return void where possible
WT-2892 hot backup can race with block truncate
WT-2896 Coverity #1362535: resource leak
WT-2897 Checkpoints can become corrupted on failure
WT-2901 Add option to disable checkpoint dirty stepdown phase
WT-2903 Reduce the impact of checkpoint scrubbing on applications
Branch: v3.2
https://github.com/mongodb/mongo/commit/7d2acd6395ec84beca34718a75371bc11f0c9f60