Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2707

dist/s_label enhancements, and error jump cleanups

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: WT2.8.0
    • Fix Version/s: WT2.9.0, 3.3.9
    • Labels:
      None
    • # Replies:
      8
    • Last comment by Customer:
      true

      Description

      When s_label determine if a jump to the error label (e.g. WT_ERR) occurs before a regular return (e.g. WT_RET), it should match other macros, like WT_RET_ERROR_OK().

      Also, within a loop, any mixing of WT_RET (and friends) and WT_ERR (and friends) is incorrect:

          WT_RET(f());
          while (cond) {
             WT_RET(g());
             WT_ERR(h());
          }
      

      This example is currently allowed, but can be detected.

      I've marked this as a bug as it will require some code fixes.

        Activity

        Hide
        keith.bostic Keith Bostic added a comment -

        Donald Anderson, I'm sure you're correct about the missing macros, macros have changed over time and the script was probably not updated.

        I'm wondering about the example, though – how do you detect that mistake, without figuring out the scope of the enclosing loop?

        Show
        keith.bostic Keith Bostic added a comment - Donald Anderson , I'm sure you're correct about the missing macros, macros have changed over time and the script was probably not updated. I'm wondering about the example, though – how do you detect that mistake, without figuring out the scope of the enclosing loop?
        Hide
        donald.anderson Donald Anderson added a comment -

        Keith Bostic, yes, I'd figure out the scope of the enclosing loop: regex isn't up to the task (at least without regex recursion), so a little python helper script will mark the loops. Turns out we only need to match the outermost loops to detect the problem, so it's just counting left/right braces and marking the right ones. I have it in perl, but I thought python would be better, since we haven't used perl anywhere yet.

        Show
        donald.anderson Donald Anderson added a comment - Keith Bostic , yes, I'd figure out the scope of the enclosing loop: regex isn't up to the task (at least without regex recursion), so a little python helper script will mark the loops. Turns out we only need to match the outermost loops to detect the problem, so it's just counting left/right braces and marking the right ones. I have it in perl, but I thought python would be better, since we haven't used perl anywhere yet.
        Hide
        keith.bostic Keith Bostic added a comment -

        Donald Anderson, I've wondered if we could get some leverage by replacing the regex and sed expressions in the dist scripts with a real Python C parser (for example, pycparser?

        The docs for pycparser indicate it's been used to write static code checkers, which seems related to this task. And more generally, we have magic shell scripts that figure out external prototypes based on pretty explicit assumptions on how a function is declared, and maybe something like pycparser would make things better.

        A more focused change might be to extend the clang static analyzer to catch this error. See How to Write a Checker in 24 Hours, it's a discussion about how to extend clang's static analyzer with your own tests. I'm less excited about this change because I think I'm the only person regularly running the clang static checker and it requires a human to look at the output, I haven't found a way to automate it, so it would be harder to integrate this into our workflow.

        Anyway, just thinking of ways we might get past creating adhoc scripts that need to be extended over time, since you're thinking about this as well.

        Show
        keith.bostic Keith Bostic added a comment - Donald Anderson , I've wondered if we could get some leverage by replacing the regex and sed expressions in the dist scripts with a real Python C parser (for example, pycparser ? The docs for pycparser indicate it's been used to write static code checkers, which seems related to this task. And more generally, we have magic shell scripts that figure out external prototypes based on pretty explicit assumptions on how a function is declared, and maybe something like pycparser would make things better. A more focused change might be to extend the clang static analyzer to catch this error. See How to Write a Checker in 24 Hours , it's a discussion about how to extend clang's static analyzer with your own tests. I'm less excited about this change because I think I'm the only person regularly running the clang static checker and it requires a human to look at the output, I haven't found a way to automate it, so it would be harder to integrate this into our workflow. Anyway, just thinking of ways we might get past creating adhoc scripts that need to be extended over time, since you're thinking about this as well.
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'ddanderson', u'name': u'Don Anderson', u'email': u'dda@mongodb.com'}

        Message: WT-2707 stricter checks in dist/s_label, and add some WT_ERR calls (#2800)

        • WT-2707 Stricter tests for label tests: match variants of WT_ERR/WT_RET,
          and do not allow mixing of WT_ERR/WT_RET in a loop.
        • Change WT_RET to WT_ERR to get proper cleanup.
        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'ddanderson', u'name': u'Don Anderson', u'email': u'dda@mongodb.com'} Message: WT-2707 stricter checks in dist/s_label, and add some WT_ERR calls (#2800) WT-2707 Stricter tests for label tests: match variants of WT_ERR/WT_RET, and do not allow mixing of WT_ERR/WT_RET in a loop. Change WT_RET to WT_ERR to get proper cleanup. Minor simplification of the error case. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/f18d9758cda5d796a9439dee0ccd202c34c4a0cf
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'ddanderson', u'name': u'Don Anderson', u'email': u'dda@mongodb.com'}

        Message: WT-2707 stricter checks in dist/s_label, and add some WT_ERR calls (#2800)

        • WT-2707 Stricter tests for label tests: match variants of WT_ERR/WT_RET,
          and do not allow mixing of WT_ERR/WT_RET in a loop.
        • Change WT_RET to WT_ERR to get proper cleanup.
        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'ddanderson', u'name': u'Don Anderson', u'email': u'dda@mongodb.com'} Message: WT-2707 stricter checks in dist/s_label, and add some WT_ERR calls (#2800) WT-2707 Stricter tests for label tests: match variants of WT_ERR/WT_RET, and do not allow mixing of WT_ERR/WT_RET in a loop. Change WT_RET to WT_ERR to get proper cleanup. Minor simplification of the error case. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/f18d9758cda5d796a9439dee0ccd202c34c4a0cf
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}

        Message: Import wiredtiger-wiredtiger-2.8.0-511-gd8fb874.tar.gz from wiredtiger branch mongodb-3.4

        ref: fb1663e..d8fb874

        SERVER-24580 Performance is poor when WiredTiger cache is full
        WT-2319 Add statistics around fsync calls
        WT-2408 Windows error translation layer
        WT-2514 log path name is an empty string.
        WT-2541 Add statistics for number of threads currently in read/write
        WT-2605 C tests focused on different schema types needed for join and other APIs
        WT-2622 unit test failed in test_cursor_random.test_cursor_random.test_cursor_random_multiple_page_records
        WT-2652 Logging code improvements
        WT-2664 Change eviction so any eviction thread can find candidates
        WT-2667 Enhance WiredTiger Evergreen testing
        WT-2668 Create join statistics that are useful and are easy to understand
        WT-2671 dump more information about the file layout in verify debug mode
        WT-2672 Handle system calls that don't set errno
        WT-2676 Don't use key size in column store in-memory splits.
        WT-2677 Fix JSON output so only printable ASCII is produced (seen on Solaris)
        WT-2682 add option to configure WiredTiger with strict compiler flags
        WT-2683 WiredTiger no longer needs to return non-zero disk sizes
        WT-2685 hazard pointer failure from clear walk
        WT-2686 logging subsystem core dump
        WT-2687 test suite should verify the exit status of the wt utility
        WT-2689 Use after free in WT_SESSION::open_cursor
        WT-2691 Use wrappers for ctype functions to avoid sign extension errors
        WT-2692 Fix race in file system example
        WT-2696 Missing log records with large updates
        WT-2698 Test/recovery hung
        WT-2702 Under high thread load, WiredTiger exceeds cache size
        WT-2704 test/format hung on bengal
        WT-2706 Test failure running random-abort recovery test
        WT-2707 dist/s_label enhancements, and error jump cleanups
        WT-2709 connection reconfigure segfault in __wt_conn_cache_pool_destroy
        WT-2710 WT_FILE_HANDLE_INMEM no longer needs an off field
        WT-2712 Coverity 1356928 and 1356929: ASSERT_SIDE_EFFECT
        WT-2713 document WT_PANIC so pluggable filesystems can panic.
        WT-2714 lint
        WT-2715 random-abort test may write partial record at the end
        WT-2720 Pull request tester not running Python suite
        WT-2722 s_label or s_label_loop false positive
        WT-2724 Eviction workers created on open exit immediately
        SERVER-24425 WiredTiger changes for MongoDB 3.3.9
        Branch: master
        https://github.com/mongodb/mongo/commit/7e3ce9cec306df5abb15c0c1a599023270cfb807

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'} Message: Import wiredtiger-wiredtiger-2.8.0-511-gd8fb874.tar.gz from wiredtiger branch mongodb-3.4 ref: fb1663e..d8fb874 SERVER-24580 Performance is poor when WiredTiger cache is full WT-2319 Add statistics around fsync calls WT-2408 Windows error translation layer WT-2514 log path name is an empty string. WT-2541 Add statistics for number of threads currently in read/write WT-2605 C tests focused on different schema types needed for join and other APIs WT-2622 unit test failed in test_cursor_random.test_cursor_random.test_cursor_random_multiple_page_records WT-2652 Logging code improvements WT-2664 Change eviction so any eviction thread can find candidates WT-2667 Enhance WiredTiger Evergreen testing WT-2668 Create join statistics that are useful and are easy to understand WT-2671 dump more information about the file layout in verify debug mode WT-2672 Handle system calls that don't set errno WT-2676 Don't use key size in column store in-memory splits. WT-2677 Fix JSON output so only printable ASCII is produced (seen on Solaris) WT-2682 add option to configure WiredTiger with strict compiler flags WT-2683 WiredTiger no longer needs to return non-zero disk sizes WT-2685 hazard pointer failure from clear walk WT-2686 logging subsystem core dump WT-2687 test suite should verify the exit status of the wt utility WT-2689 Use after free in WT_SESSION::open_cursor WT-2691 Use wrappers for ctype functions to avoid sign extension errors WT-2692 Fix race in file system example WT-2696 Missing log records with large updates WT-2698 Test/recovery hung WT-2702 Under high thread load, WiredTiger exceeds cache size WT-2704 test/format hung on bengal WT-2706 Test failure running random-abort recovery test WT-2707 dist/s_label enhancements, and error jump cleanups WT-2709 connection reconfigure segfault in __wt_conn_cache_pool_destroy WT-2710 WT_FILE_HANDLE_INMEM no longer needs an off field WT-2712 Coverity 1356928 and 1356929: ASSERT_SIDE_EFFECT WT-2713 document WT_PANIC so pluggable filesystems can panic. WT-2714 lint WT-2715 random-abort test may write partial record at the end WT-2720 Pull request tester not running Python suite WT-2722 s_label or s_label_loop false positive WT-2724 Eviction workers created on open exit immediately SERVER-24425 WiredTiger changes for MongoDB 3.3.9 Branch: master https://github.com/mongodb/mongo/commit/7e3ce9cec306df5abb15c0c1a599023270cfb807
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}

        Message: Import wiredtiger-wiredtiger-2.8.0-592-g848e5f5.tar.gz from wiredtiger branch mongodb-3.2

        ref: 8b7110b..848e5f5
        for: 3.2.9

        This commit replaces a number of previous backports with the original
        change, so some of the tickets listed below may have been resolved in
        previous versions already as a backport to the v3.2 branch. For example,
        SERVER-24580 includes change a63e21b8 as a cherry-pick of 1f4aaa44, but
        this commit replaces that change with 1f4aaa44 itself.

        SERVER-23659 Provide useful message when wiredTigerJournalCompressor is changed
        SERVER-23661 $sample takes disproportionately long time on newly created collection
        SERVER-24306 40-second journaling stall from "log files prepared" to checkpoint
        SERVER-24580 Improve performance when WiredTiger cache is full
        WT-2103 Add incremental backup testing to format
        WT-2223 Add stress testing for in-memory
        WT-2268 JSON load incorrect with UNICODE input
        WT-2319 Add statistics around fsync calls
        WT-2325 Fix an incomplete comment
        WT-2343 Assert we don't remove or rename when backup cursor is open
        WT-2349 Add ability to open databases read-only
        WT-2359 WiredTiger with Python will hang if a calloc failure occurs during __wt_connection_close
        WT-2360 Allow disjunctions and combinations of operations in join cursors
        WT-2408 Windows error translation layer
        WT-2446 Estimate WT cache hit ratio
        WT-2450 Salvage releases pages, then explicitly evicts them.
        WT-2453 Throughput drop in wtperf evict Jenkins tests
        WT-2479 Dump utility discards table config (JSON)
        WT-2491 The dhandle close_lock isn't valuable at the moment
        WT-2504 Should READONLY always read basecfg file?
        WT-2505 Review clang analyzer warnings
        WT-2508 Test programs should remove test directories on the "clean" target
        WT-2514 Log path name is an empty string.
        WT-2518 LSM checkpoint handle acquisition optimization
        WT-2520 WT_SESSION::verify should not alter tables
        WT-2526 Mixing and matching readonly and read/write handles
        WT-2535 Extend test/format to test for transactions reading their writes
        WT-2537 Cannot open DB written by WT2.6.1 with WT2.8.0 due to WT_NOTFOUND on recovery
        WT-2539 Implement file streaming above pluggable filesystems
        WT-2540 Separate stream and file handle methods
        WT-2541 Add statistics for number of threads currently in read/write
        WT-2542 Fixed-length column store reconciliation overwrites original values
        WT-2544 Fix eviction statistics when clear is configured
        WT-2546 Eviction server not help evict pages sometimes
        WT-2547 Add 1-eviction-worker jobs to Jenkins
        WT-2548 Cap the amount of data handed to raw compression.
        WT-2549 joins using recno keys return no values
        WT-2550 java ex_schema example fails
        WT-2552 Public API for pluggable filesystems
        WT-2553 Document in-memory configuration and WT_CACHE_FULL error return
        WT-2554 Implement a framework for adding C test cases
        WT-2556 Typo in the Java example code
        WT-2557 format test program should discard log files after incremental backup
        WT-2558 WT_PAGE structure reorganization
        WT-2559 Windows segfault in logging code
        WT-2560 Stuck trying to update oldest transaction ID
        WT-2562 Reconfig02 test failing sometimes on PPC
        WT-2565 Item 3573 on page at [write-check] is a corrupted cell
        WT-2566 All lock operations should be barriers
        WT-2567 Segfault in test/format log truncate
        WT-2568 Java PackTest.java compilation error
        WT-2569 Win_handle_read should always call GetLastError on error
        WT-2570 Minor lint cleanups.
        WT-2571 Join code cleanup
        WT-2572 Don't select an in-memory format run if incompatible options configured
        WT-2573 Free of stack-allocated WT_REF
        WT-2574 Format doesn't free all allocated configure memory
        WT-2576 Variable-length column-store out-of-order return
        WT-2577 Core dump discarding non-existent addresses
        WT-2579 In-memory configurations break debugging support
        WT-2580 Potential SWIG naming conflict in Java
        WT-2581 Assert multi->disk_image == NULL
        WT-2582 Cache eviction server error: WT_RESTART
        WT-2583 Incremental backup can prevent future recovery
        WT-2584 Don't use periods in error messages
        WT-2586 Remove ex_config.c until config cursors are supported
        WT-2589 Check stats using WT_STAT_SET and 'clear' usage
        WT-2592 Joins using non-recno key types not working
        WT-2593 Disk full with pre-allocated log files
        WT-2595 Fix compiler warning in packing tests
        WT-2597 JSON dump needs LSM tests
        WT-2598 In-memory FS needs fast lookup on file names
        WT-2599 Split out the checksum code from the support directory
        WT-2600 Clean up test program #includes
        WT-2602 LSM stress hangs with very large uncompressed pages
        WT-2605 C tests focused on different schema types needed for join and other APIs
        WT-2609 Incorrect "skips API_END call" error.
        WT-2610 Investigate eviction hazard pointer check
        WT-2611 wtperf and/or WT_CONFIG_PARSER do not allow escaped double quotes
        WT-2612 The dist/s_prototypes script is creating a debugging file xxx.
        WT-2613 Windows build failing with a C4100 error
        WT-2615 Enabling checkpoints in test/format leads to reduced concurrency
        WT-2616 In-memory deadlock getting size
        WT-2617 Bug in pluggable file system example
        WT-2621 WiredTiger fails to compile on MSVC 2013
        WT-2622 Unit test failed in test_cursor_random.test_cursor_random.test_cursor_random_multiple_page_records
        WT-2624 snprintf unsupported on MSVC 2013
        WT-2626 snprintf errors with macro redefinition on MSVC2015
        WT-2627 Coverity complaints
        WT-2628 Reconciliation can return without unlocking the page lock
        WT-2629 Introduction of ppc64le crc32c assembly file has made the stack executable
        WT-2630 Rename WT_FSTREAM methods to avoid using C99 reserved names
        WT-2632 Test format failures due to "checkpoints cannot be dropped when in-use"
        WT-2637 The file-extension documentation doesn't cover not-supported cases
        WT-2644 'wt load -r' (rename) fails with LSM
        WT-2646 Split the lock_wait flag into two, adding a checkpoint_wait flag
        WT-2651 Coverity 1355591 resource leak
        WT-2652 Logging code improvements
        WT-2653 The custom file-system example should show device configuration
        WT-2656 Builds failing on GCC 4.7 builder
        WT-2658 Only include PPC-specific files in PPC builds
        WT-2659 csuite tests, assorted lint and cleanup.
        WT-2660 Hang between eviction and connection close
        WT-2661 Coverity failures: 1356050-1356053
        WT-2662 For internal spell checking, strip out double quote literals, they confuse aspell
        WT-2664 Change eviction so any eviction thread can find candidates
        WT-2667 Enhance WiredTiger Evergreen testing
        WT-2668 Create join statistics that are useful and are easy to understand
        WT-2671 Dump more information about the file layout in verify debug mode
        WT-2672 Handle system calls that don't set errno
        WT-2673 Stop automatically increasing memory page max
        WT-2674 Simplify metadata file check
        WT-2676 Don't use key size in column store in-memory splits.
        WT-2677 Fix JSON output so only printable ASCII is produced (seen on Solaris)
        WT-2682 Add option to configure WiredTiger with strict compiler flags
        WT-2683 WiredTiger no longer needs to return non-zero disk sizes
        WT-2685 Hazard pointer failure from clear walk
        WT-2686 Logging subsystem core dump
        WT-2687 Test suite should verify the exit status of the wt utility
        WT-2689 Use after free in WT_SESSION::open_cursor
        WT-2691 Use wrappers for ctype functions to avoid sign extension errors
        WT-2692 Fix race in file system example
        WT-2696 Race condition on unclean shutdown may miss log records with large updates
        WT-2698 Test/recovery hung
        WT-2702 Under high thread load, WiredTiger exceeds cache size
        WT-2704 Test/format hung on bengal
        WT-2706 Race condition on log file switch can cause missing log records
        WT-2707 dist/s_label enhancements, and error jump cleanups
        WT-2709 Connection reconfigure segfault in __wt_conn_cache_pool_destroy
        WT-2710 WT_FILE_HANDLE_INMEM no longer needs an off field
        WT-2712 Coverity 1356928 and 1356929: ASSERT_SIDE_EFFECT
        WT-2713 Document WT_PANIC so pluggable filesystems can panic.
        WT-2714 Lint
        WT-2715 random-abort test may write partial record at the end
        WT-2720 Pull request tester not running Python suite
        WT-2722 s_label or s_label_loop false positive
        WT-2724 Eviction workers created on open exit immediately
        WT-2763 Unit test test_intpack failing on OSX
        Branch: v3.2
        https://github.com/mongodb/mongo/commit/385faa6ca72ad599d92d3948f1a2033d908e0226

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'} Message: Import wiredtiger-wiredtiger-2.8.0-592-g848e5f5.tar.gz from wiredtiger branch mongodb-3.2 ref: 8b7110b..848e5f5 for: 3.2.9 This commit replaces a number of previous backports with the original change, so some of the tickets listed below may have been resolved in previous versions already as a backport to the v3.2 branch. For example, SERVER-24580 includes change a63e21b8 as a cherry-pick of 1f4aaa44, but this commit replaces that change with 1f4aaa44 itself. SERVER-23659 Provide useful message when wiredTigerJournalCompressor is changed SERVER-23661 $sample takes disproportionately long time on newly created collection SERVER-24306 40-second journaling stall from "log files prepared" to checkpoint SERVER-24580 Improve performance when WiredTiger cache is full WT-2103 Add incremental backup testing to format WT-2223 Add stress testing for in-memory WT-2268 JSON load incorrect with UNICODE input WT-2319 Add statistics around fsync calls WT-2325 Fix an incomplete comment WT-2343 Assert we don't remove or rename when backup cursor is open WT-2349 Add ability to open databases read-only WT-2359 WiredTiger with Python will hang if a calloc failure occurs during __wt_connection_close WT-2360 Allow disjunctions and combinations of operations in join cursors WT-2408 Windows error translation layer WT-2446 Estimate WT cache hit ratio WT-2450 Salvage releases pages, then explicitly evicts them. WT-2453 Throughput drop in wtperf evict Jenkins tests WT-2479 Dump utility discards table config (JSON) WT-2491 The dhandle close_lock isn't valuable at the moment WT-2504 Should READONLY always read basecfg file? WT-2505 Review clang analyzer warnings WT-2508 Test programs should remove test directories on the "clean" target WT-2514 Log path name is an empty string. WT-2518 LSM checkpoint handle acquisition optimization WT-2520 WT_SESSION::verify should not alter tables WT-2526 Mixing and matching readonly and read/write handles WT-2535 Extend test/format to test for transactions reading their writes WT-2537 Cannot open DB written by WT2.6.1 with WT2.8.0 due to WT_NOTFOUND on recovery WT-2539 Implement file streaming above pluggable filesystems WT-2540 Separate stream and file handle methods WT-2541 Add statistics for number of threads currently in read/write WT-2542 Fixed-length column store reconciliation overwrites original values WT-2544 Fix eviction statistics when clear is configured WT-2546 Eviction server not help evict pages sometimes WT-2547 Add 1-eviction-worker jobs to Jenkins WT-2548 Cap the amount of data handed to raw compression. WT-2549 joins using recno keys return no values WT-2550 java ex_schema example fails WT-2552 Public API for pluggable filesystems WT-2553 Document in-memory configuration and WT_CACHE_FULL error return WT-2554 Implement a framework for adding C test cases WT-2556 Typo in the Java example code WT-2557 format test program should discard log files after incremental backup WT-2558 WT_PAGE structure reorganization WT-2559 Windows segfault in logging code WT-2560 Stuck trying to update oldest transaction ID WT-2562 Reconfig02 test failing sometimes on PPC WT-2565 Item 3573 on page at [write-check] is a corrupted cell WT-2566 All lock operations should be barriers WT-2567 Segfault in test/format log truncate WT-2568 Java PackTest.java compilation error WT-2569 Win_handle_read should always call GetLastError on error WT-2570 Minor lint cleanups. WT-2571 Join code cleanup WT-2572 Don't select an in-memory format run if incompatible options configured WT-2573 Free of stack-allocated WT_REF WT-2574 Format doesn't free all allocated configure memory WT-2576 Variable-length column-store out-of-order return WT-2577 Core dump discarding non-existent addresses WT-2579 In-memory configurations break debugging support WT-2580 Potential SWIG naming conflict in Java WT-2581 Assert multi->disk_image == NULL WT-2582 Cache eviction server error: WT_RESTART WT-2583 Incremental backup can prevent future recovery WT-2584 Don't use periods in error messages WT-2586 Remove ex_config.c until config cursors are supported WT-2589 Check stats using WT_STAT_SET and 'clear' usage WT-2592 Joins using non-recno key types not working WT-2593 Disk full with pre-allocated log files WT-2595 Fix compiler warning in packing tests WT-2597 JSON dump needs LSM tests WT-2598 In-memory FS needs fast lookup on file names WT-2599 Split out the checksum code from the support directory WT-2600 Clean up test program #includes WT-2602 LSM stress hangs with very large uncompressed pages WT-2605 C tests focused on different schema types needed for join and other APIs WT-2609 Incorrect "skips API_END call" error. WT-2610 Investigate eviction hazard pointer check WT-2611 wtperf and/or WT_CONFIG_PARSER do not allow escaped double quotes WT-2612 The dist/s_prototypes script is creating a debugging file xxx. WT-2613 Windows build failing with a C4100 error WT-2615 Enabling checkpoints in test/format leads to reduced concurrency WT-2616 In-memory deadlock getting size WT-2617 Bug in pluggable file system example WT-2621 WiredTiger fails to compile on MSVC 2013 WT-2622 Unit test failed in test_cursor_random.test_cursor_random.test_cursor_random_multiple_page_records WT-2624 snprintf unsupported on MSVC 2013 WT-2626 snprintf errors with macro redefinition on MSVC2015 WT-2627 Coverity complaints WT-2628 Reconciliation can return without unlocking the page lock WT-2629 Introduction of ppc64le crc32c assembly file has made the stack executable WT-2630 Rename WT_FSTREAM methods to avoid using C99 reserved names WT-2632 Test format failures due to "checkpoints cannot be dropped when in-use" WT-2637 The file-extension documentation doesn't cover not-supported cases WT-2644 'wt load -r' (rename) fails with LSM WT-2646 Split the lock_wait flag into two, adding a checkpoint_wait flag WT-2651 Coverity 1355591 resource leak WT-2652 Logging code improvements WT-2653 The custom file-system example should show device configuration WT-2656 Builds failing on GCC 4.7 builder WT-2658 Only include PPC-specific files in PPC builds WT-2659 csuite tests, assorted lint and cleanup. WT-2660 Hang between eviction and connection close WT-2661 Coverity failures: 1356050-1356053 WT-2662 For internal spell checking, strip out double quote literals, they confuse aspell WT-2664 Change eviction so any eviction thread can find candidates WT-2667 Enhance WiredTiger Evergreen testing WT-2668 Create join statistics that are useful and are easy to understand WT-2671 Dump more information about the file layout in verify debug mode WT-2672 Handle system calls that don't set errno WT-2673 Stop automatically increasing memory page max WT-2674 Simplify metadata file check WT-2676 Don't use key size in column store in-memory splits. WT-2677 Fix JSON output so only printable ASCII is produced (seen on Solaris) WT-2682 Add option to configure WiredTiger with strict compiler flags WT-2683 WiredTiger no longer needs to return non-zero disk sizes WT-2685 Hazard pointer failure from clear walk WT-2686 Logging subsystem core dump WT-2687 Test suite should verify the exit status of the wt utility WT-2689 Use after free in WT_SESSION::open_cursor WT-2691 Use wrappers for ctype functions to avoid sign extension errors WT-2692 Fix race in file system example WT-2696 Race condition on unclean shutdown may miss log records with large updates WT-2698 Test/recovery hung WT-2702 Under high thread load, WiredTiger exceeds cache size WT-2704 Test/format hung on bengal WT-2706 Race condition on log file switch can cause missing log records WT-2707 dist/s_label enhancements, and error jump cleanups WT-2709 Connection reconfigure segfault in __wt_conn_cache_pool_destroy WT-2710 WT_FILE_HANDLE_INMEM no longer needs an off field WT-2712 Coverity 1356928 and 1356929: ASSERT_SIDE_EFFECT WT-2713 Document WT_PANIC so pluggable filesystems can panic. WT-2714 Lint WT-2715 random-abort test may write partial record at the end WT-2720 Pull request tester not running Python suite WT-2722 s_label or s_label_loop false positive WT-2724 Eviction workers created on open exit immediately WT-2763 Unit test test_intpack failing on OSX Branch: v3.2 https://github.com/mongodb/mongo/commit/385faa6ca72ad599d92d3948f1a2033d908e0226
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}

        Message: Import wiredtiger-wiredtiger-2.8.0-592-g848e5f5.tar.gz from wiredtiger branch mongodb-3.2

        ref: 8b7110b..848e5f5
        for: 3.2.9

        This commit replaces a number of previous backports with the original
        change, so some of the tickets listed below may have been resolved in
        previous versions already as a backport to the v3.2 branch. For example,
        SERVER-24580 includes change a63e21b8 as a cherry-pick of 1f4aaa44, but
        this commit replaces that change with 1f4aaa44 itself.

        SERVER-23659 Provide useful message when wiredTigerJournalCompressor is changed
        SERVER-23661 $sample takes disproportionately long time on newly created collection
        SERVER-24306 40-second journaling stall from "log files prepared" to checkpoint
        SERVER-24580 Improve performance when WiredTiger cache is full
        WT-2103 Add incremental backup testing to format
        WT-2223 Add stress testing for in-memory
        WT-2268 JSON load incorrect with UNICODE input
        WT-2319 Add statistics around fsync calls
        WT-2325 Fix an incomplete comment
        WT-2343 Assert we don't remove or rename when backup cursor is open
        WT-2349 Add ability to open databases read-only
        WT-2359 WiredTiger with Python will hang if a calloc failure occurs during __wt_connection_close
        WT-2360 Allow disjunctions and combinations of operations in join cursors
        WT-2408 Windows error translation layer
        WT-2446 Estimate WT cache hit ratio
        WT-2450 Salvage releases pages, then explicitly evicts them.
        WT-2453 Throughput drop in wtperf evict Jenkins tests
        WT-2479 Dump utility discards table config (JSON)
        WT-2491 The dhandle close_lock isn't valuable at the moment
        WT-2504 Should READONLY always read basecfg file?
        WT-2505 Review clang analyzer warnings
        WT-2508 Test programs should remove test directories on the "clean" target
        WT-2514 Log path name is an empty string.
        WT-2518 LSM checkpoint handle acquisition optimization
        WT-2520 WT_SESSION::verify should not alter tables
        WT-2526 Mixing and matching readonly and read/write handles
        WT-2535 Extend test/format to test for transactions reading their writes
        WT-2537 Cannot open DB written by WT2.6.1 with WT2.8.0 due to WT_NOTFOUND on recovery
        WT-2539 Implement file streaming above pluggable filesystems
        WT-2540 Separate stream and file handle methods
        WT-2541 Add statistics for number of threads currently in read/write
        WT-2542 Fixed-length column store reconciliation overwrites original values
        WT-2544 Fix eviction statistics when clear is configured
        WT-2546 Eviction server not help evict pages sometimes
        WT-2547 Add 1-eviction-worker jobs to Jenkins
        WT-2548 Cap the amount of data handed to raw compression.
        WT-2549 joins using recno keys return no values
        WT-2550 java ex_schema example fails
        WT-2552 Public API for pluggable filesystems
        WT-2553 Document in-memory configuration and WT_CACHE_FULL error return
        WT-2554 Implement a framework for adding C test cases
        WT-2556 Typo in the Java example code
        WT-2557 format test program should discard log files after incremental backup
        WT-2558 WT_PAGE structure reorganization
        WT-2559 Windows segfault in logging code
        WT-2560 Stuck trying to update oldest transaction ID
        WT-2562 Reconfig02 test failing sometimes on PPC
        WT-2565 Item 3573 on page at [write-check] is a corrupted cell
        WT-2566 All lock operations should be barriers
        WT-2567 Segfault in test/format log truncate
        WT-2568 Java PackTest.java compilation error
        WT-2569 Win_handle_read should always call GetLastError on error
        WT-2570 Minor lint cleanups.
        WT-2571 Join code cleanup
        WT-2572 Don't select an in-memory format run if incompatible options configured
        WT-2573 Free of stack-allocated WT_REF
        WT-2574 Format doesn't free all allocated configure memory
        WT-2576 Variable-length column-store out-of-order return
        WT-2577 Core dump discarding non-existent addresses
        WT-2579 In-memory configurations break debugging support
        WT-2580 Potential SWIG naming conflict in Java
        WT-2581 Assert multi->disk_image == NULL
        WT-2582 Cache eviction server error: WT_RESTART
        WT-2583 Incremental backup can prevent future recovery
        WT-2584 Don't use periods in error messages
        WT-2586 Remove ex_config.c until config cursors are supported
        WT-2589 Check stats using WT_STAT_SET and 'clear' usage
        WT-2592 Joins using non-recno key types not working
        WT-2593 Disk full with pre-allocated log files
        WT-2595 Fix compiler warning in packing tests
        WT-2597 JSON dump needs LSM tests
        WT-2598 In-memory FS needs fast lookup on file names
        WT-2599 Split out the checksum code from the support directory
        WT-2600 Clean up test program #includes
        WT-2602 LSM stress hangs with very large uncompressed pages
        WT-2605 C tests focused on different schema types needed for join and other APIs
        WT-2609 Incorrect "skips API_END call" error.
        WT-2610 Investigate eviction hazard pointer check
        WT-2611 wtperf and/or WT_CONFIG_PARSER do not allow escaped double quotes
        WT-2612 The dist/s_prototypes script is creating a debugging file xxx.
        WT-2613 Windows build failing with a C4100 error
        WT-2615 Enabling checkpoints in test/format leads to reduced concurrency
        WT-2616 In-memory deadlock getting size
        WT-2617 Bug in pluggable file system example
        WT-2621 WiredTiger fails to compile on MSVC 2013
        WT-2622 Unit test failed in test_cursor_random.test_cursor_random.test_cursor_random_multiple_page_records
        WT-2624 snprintf unsupported on MSVC 2013
        WT-2626 snprintf errors with macro redefinition on MSVC2015
        WT-2627 Coverity complaints
        WT-2628 Reconciliation can return without unlocking the page lock
        WT-2629 Introduction of ppc64le crc32c assembly file has made the stack executable
        WT-2630 Rename WT_FSTREAM methods to avoid using C99 reserved names
        WT-2632 Test format failures due to "checkpoints cannot be dropped when in-use"
        WT-2637 The file-extension documentation doesn't cover not-supported cases
        WT-2644 'wt load -r' (rename) fails with LSM
        WT-2646 Split the lock_wait flag into two, adding a checkpoint_wait flag
        WT-2651 Coverity 1355591 resource leak
        WT-2652 Logging code improvements
        WT-2653 The custom file-system example should show device configuration
        WT-2656 Builds failing on GCC 4.7 builder
        WT-2658 Only include PPC-specific files in PPC builds
        WT-2659 csuite tests, assorted lint and cleanup.
        WT-2660 Hang between eviction and connection close
        WT-2661 Coverity failures: 1356050-1356053
        WT-2662 For internal spell checking, strip out double quote literals, they confuse aspell
        WT-2664 Change eviction so any eviction thread can find candidates
        WT-2667 Enhance WiredTiger Evergreen testing
        WT-2668 Create join statistics that are useful and are easy to understand
        WT-2671 Dump more information about the file layout in verify debug mode
        WT-2672 Handle system calls that don't set errno
        WT-2673 Stop automatically increasing memory page max
        WT-2674 Simplify metadata file check
        WT-2676 Don't use key size in column store in-memory splits.
        WT-2677 Fix JSON output so only printable ASCII is produced (seen on Solaris)
        WT-2682 Add option to configure WiredTiger with strict compiler flags
        WT-2683 WiredTiger no longer needs to return non-zero disk sizes
        WT-2685 Hazard pointer failure from clear walk
        WT-2686 Logging subsystem core dump
        WT-2687 Test suite should verify the exit status of the wt utility
        WT-2689 Use after free in WT_SESSION::open_cursor
        WT-2691 Use wrappers for ctype functions to avoid sign extension errors
        WT-2692 Fix race in file system example
        WT-2696 Race condition on unclean shutdown may miss log records with large updates
        WT-2698 Test/recovery hung
        WT-2702 Under high thread load, WiredTiger exceeds cache size
        WT-2704 Test/format hung on bengal
        WT-2706 Race condition on log file switch can cause missing log records
        WT-2707 dist/s_label enhancements, and error jump cleanups
        WT-2709 Connection reconfigure segfault in __wt_conn_cache_pool_destroy
        WT-2710 WT_FILE_HANDLE_INMEM no longer needs an off field
        WT-2712 Coverity 1356928 and 1356929: ASSERT_SIDE_EFFECT
        WT-2713 Document WT_PANIC so pluggable filesystems can panic.
        WT-2714 Lint
        WT-2715 random-abort test may write partial record at the end
        WT-2720 Pull request tester not running Python suite
        WT-2722 s_label or s_label_loop false positive
        WT-2724 Eviction workers created on open exit immediately
        WT-2763 Unit test test_intpack failing on OSX
        Branch: v3.2
        https://github.com/mongodb/mongo/commit/385faa6ca72ad599d92d3948f1a2033d908e0226

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'} Message: Import wiredtiger-wiredtiger-2.8.0-592-g848e5f5.tar.gz from wiredtiger branch mongodb-3.2 ref: 8b7110b..848e5f5 for: 3.2.9 This commit replaces a number of previous backports with the original change, so some of the tickets listed below may have been resolved in previous versions already as a backport to the v3.2 branch. For example, SERVER-24580 includes change a63e21b8 as a cherry-pick of 1f4aaa44, but this commit replaces that change with 1f4aaa44 itself. SERVER-23659 Provide useful message when wiredTigerJournalCompressor is changed SERVER-23661 $sample takes disproportionately long time on newly created collection SERVER-24306 40-second journaling stall from "log files prepared" to checkpoint SERVER-24580 Improve performance when WiredTiger cache is full WT-2103 Add incremental backup testing to format WT-2223 Add stress testing for in-memory WT-2268 JSON load incorrect with UNICODE input WT-2319 Add statistics around fsync calls WT-2325 Fix an incomplete comment WT-2343 Assert we don't remove or rename when backup cursor is open WT-2349 Add ability to open databases read-only WT-2359 WiredTiger with Python will hang if a calloc failure occurs during __wt_connection_close WT-2360 Allow disjunctions and combinations of operations in join cursors WT-2408 Windows error translation layer WT-2446 Estimate WT cache hit ratio WT-2450 Salvage releases pages, then explicitly evicts them. WT-2453 Throughput drop in wtperf evict Jenkins tests WT-2479 Dump utility discards table config (JSON) WT-2491 The dhandle close_lock isn't valuable at the moment WT-2504 Should READONLY always read basecfg file? WT-2505 Review clang analyzer warnings WT-2508 Test programs should remove test directories on the "clean" target WT-2514 Log path name is an empty string. WT-2518 LSM checkpoint handle acquisition optimization WT-2520 WT_SESSION::verify should not alter tables WT-2526 Mixing and matching readonly and read/write handles WT-2535 Extend test/format to test for transactions reading their writes WT-2537 Cannot open DB written by WT2.6.1 with WT2.8.0 due to WT_NOTFOUND on recovery WT-2539 Implement file streaming above pluggable filesystems WT-2540 Separate stream and file handle methods WT-2541 Add statistics for number of threads currently in read/write WT-2542 Fixed-length column store reconciliation overwrites original values WT-2544 Fix eviction statistics when clear is configured WT-2546 Eviction server not help evict pages sometimes WT-2547 Add 1-eviction-worker jobs to Jenkins WT-2548 Cap the amount of data handed to raw compression. WT-2549 joins using recno keys return no values WT-2550 java ex_schema example fails WT-2552 Public API for pluggable filesystems WT-2553 Document in-memory configuration and WT_CACHE_FULL error return WT-2554 Implement a framework for adding C test cases WT-2556 Typo in the Java example code WT-2557 format test program should discard log files after incremental backup WT-2558 WT_PAGE structure reorganization WT-2559 Windows segfault in logging code WT-2560 Stuck trying to update oldest transaction ID WT-2562 Reconfig02 test failing sometimes on PPC WT-2565 Item 3573 on page at [write-check] is a corrupted cell WT-2566 All lock operations should be barriers WT-2567 Segfault in test/format log truncate WT-2568 Java PackTest.java compilation error WT-2569 Win_handle_read should always call GetLastError on error WT-2570 Minor lint cleanups. WT-2571 Join code cleanup WT-2572 Don't select an in-memory format run if incompatible options configured WT-2573 Free of stack-allocated WT_REF WT-2574 Format doesn't free all allocated configure memory WT-2576 Variable-length column-store out-of-order return WT-2577 Core dump discarding non-existent addresses WT-2579 In-memory configurations break debugging support WT-2580 Potential SWIG naming conflict in Java WT-2581 Assert multi->disk_image == NULL WT-2582 Cache eviction server error: WT_RESTART WT-2583 Incremental backup can prevent future recovery WT-2584 Don't use periods in error messages WT-2586 Remove ex_config.c until config cursors are supported WT-2589 Check stats using WT_STAT_SET and 'clear' usage WT-2592 Joins using non-recno key types not working WT-2593 Disk full with pre-allocated log files WT-2595 Fix compiler warning in packing tests WT-2597 JSON dump needs LSM tests WT-2598 In-memory FS needs fast lookup on file names WT-2599 Split out the checksum code from the support directory WT-2600 Clean up test program #includes WT-2602 LSM stress hangs with very large uncompressed pages WT-2605 C tests focused on different schema types needed for join and other APIs WT-2609 Incorrect "skips API_END call" error. WT-2610 Investigate eviction hazard pointer check WT-2611 wtperf and/or WT_CONFIG_PARSER do not allow escaped double quotes WT-2612 The dist/s_prototypes script is creating a debugging file xxx. WT-2613 Windows build failing with a C4100 error WT-2615 Enabling checkpoints in test/format leads to reduced concurrency WT-2616 In-memory deadlock getting size WT-2617 Bug in pluggable file system example WT-2621 WiredTiger fails to compile on MSVC 2013 WT-2622 Unit test failed in test_cursor_random.test_cursor_random.test_cursor_random_multiple_page_records WT-2624 snprintf unsupported on MSVC 2013 WT-2626 snprintf errors with macro redefinition on MSVC2015 WT-2627 Coverity complaints WT-2628 Reconciliation can return without unlocking the page lock WT-2629 Introduction of ppc64le crc32c assembly file has made the stack executable WT-2630 Rename WT_FSTREAM methods to avoid using C99 reserved names WT-2632 Test format failures due to "checkpoints cannot be dropped when in-use" WT-2637 The file-extension documentation doesn't cover not-supported cases WT-2644 'wt load -r' (rename) fails with LSM WT-2646 Split the lock_wait flag into two, adding a checkpoint_wait flag WT-2651 Coverity 1355591 resource leak WT-2652 Logging code improvements WT-2653 The custom file-system example should show device configuration WT-2656 Builds failing on GCC 4.7 builder WT-2658 Only include PPC-specific files in PPC builds WT-2659 csuite tests, assorted lint and cleanup. WT-2660 Hang between eviction and connection close WT-2661 Coverity failures: 1356050-1356053 WT-2662 For internal spell checking, strip out double quote literals, they confuse aspell WT-2664 Change eviction so any eviction thread can find candidates WT-2667 Enhance WiredTiger Evergreen testing WT-2668 Create join statistics that are useful and are easy to understand WT-2671 Dump more information about the file layout in verify debug mode WT-2672 Handle system calls that don't set errno WT-2673 Stop automatically increasing memory page max WT-2674 Simplify metadata file check WT-2676 Don't use key size in column store in-memory splits. WT-2677 Fix JSON output so only printable ASCII is produced (seen on Solaris) WT-2682 Add option to configure WiredTiger with strict compiler flags WT-2683 WiredTiger no longer needs to return non-zero disk sizes WT-2685 Hazard pointer failure from clear walk WT-2686 Logging subsystem core dump WT-2687 Test suite should verify the exit status of the wt utility WT-2689 Use after free in WT_SESSION::open_cursor WT-2691 Use wrappers for ctype functions to avoid sign extension errors WT-2692 Fix race in file system example WT-2696 Race condition on unclean shutdown may miss log records with large updates WT-2698 Test/recovery hung WT-2702 Under high thread load, WiredTiger exceeds cache size WT-2704 Test/format hung on bengal WT-2706 Race condition on log file switch can cause missing log records WT-2707 dist/s_label enhancements, and error jump cleanups WT-2709 Connection reconfigure segfault in __wt_conn_cache_pool_destroy WT-2710 WT_FILE_HANDLE_INMEM no longer needs an off field WT-2712 Coverity 1356928 and 1356929: ASSERT_SIDE_EFFECT WT-2713 Document WT_PANIC so pluggable filesystems can panic. WT-2714 Lint WT-2715 random-abort test may write partial record at the end WT-2720 Pull request tester not running Python suite WT-2722 s_label or s_label_loop false positive WT-2724 Eviction workers created on open exit immediately WT-2763 Unit test test_intpack failing on OSX Branch: v3.2 https://github.com/mongodb/mongo/commit/385faa6ca72ad599d92d3948f1a2033d908e0226

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Days since reply:
              39 weeks, 3 days ago
              Date of 1st Reply: