Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2933

Fix a race between named snapshots and checkpoints

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.9.0, 3.2.11, 3.4.0-rc1
    • Labels:
      None
    • # Replies:
      9
    • Last comment by Customer:
      true

      Description

      We have seen this assertion in testing:

      [js_test:read_committed_after_rollback] 2016-08-18T00:55:35.560+0000 d20015| 2016-08-18T00:55:35.559+0000 E STORAGE  [conn62] WiredTiger (0) [1471481735:559713][16909:0x11300b000], WT_SESSION.rollback_transaction: src/third_party/wiredtiger/src/txn/txn.c, 101: txn_state->snap_min == WT_TXN_NONE || session->txn.isolation == WT_ISO_READ_UNCOMMITTED || !__wt_txn_visible_all(session, txn_state->snap_min)
      [js_test:read_committed_after_rollback] 2016-08-18T00:55:35.560+0000 d20015| 2016-08-18T00:55:35.559+0000 E STORAGE  [conn62] WiredTiger (0) [1471481735:559882][16909:0x11300b000], WT_SESSION.rollback_transaction: aborting WiredTiger library
      

      I believe that this transaction was started on a named snapshot, and the snapshot was dropped before the transaction was rolled back.

      Write a WT test case for this (create a named snapshot, start a transaction on it, do lots of additional transactions to move the oldest ID forward, try to commit / rollback the transaction and reproduce the assertion).

      The fix is to publish the snap_min for named snapshots into txn_global->states.

        Activity

        Hide
        alexander.gorrod Alexander Gorrod added a comment -

        The changes in WT-2798 introduced a bug into the allocation of named snapshots when a checkpoint is running. This bug should not have led to invalid reads or data corruption but violated some sanity checking in diagnostic builds.

        Show
        alexander.gorrod Alexander Gorrod added a comment - The changes in WT-2798 introduced a bug into the allocation of named snapshots when a checkpoint is running. This bug should not have led to invalid reads or data corruption but violated some sanity checking in diagnostic builds.
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2933 Fix a race between named snapshots and checkpoints. (#3068)
        Branch: develop
        https://github.com/wiredtiger/wiredtiger/commit/213053e190333eecf91556875317a44ac7f283f6

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2933 Fix a race between named snapshots and checkpoints. (#3068) Branch: develop https://github.com/wiredtiger/wiredtiger/commit/213053e190333eecf91556875317a44ac7f283f6
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2933 Fix an uninitialized variable warning. (#3077)
        Branch: develop
        https://github.com/wiredtiger/wiredtiger/commit/19445fbfdfb81cefcb4c9f2a67b0fba360aaa515

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2933 Fix an uninitialized variable warning. (#3077) Branch: develop https://github.com/wiredtiger/wiredtiger/commit/19445fbfdfb81cefcb4c9f2a67b0fba360aaa515
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2933 Fix a race between named snapshots and checkpoints. (#3068)
        Branch: mongodb-3.4
        https://github.com/wiredtiger/wiredtiger/commit/213053e190333eecf91556875317a44ac7f283f6

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2933 Fix a race between named snapshots and checkpoints. (#3068) Branch: mongodb-3.4 https://github.com/wiredtiger/wiredtiger/commit/213053e190333eecf91556875317a44ac7f283f6
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2933 Fix an uninitialized variable warning. (#3077)
        Branch: mongodb-3.4
        https://github.com/wiredtiger/wiredtiger/commit/19445fbfdfb81cefcb4c9f2a67b0fba360aaa515

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2933 Fix an uninitialized variable warning. (#3077) Branch: mongodb-3.4 https://github.com/wiredtiger/wiredtiger/commit/19445fbfdfb81cefcb4c9f2a67b0fba360aaa515
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

        Message: Import wiredtiger: 6a31c2118cce88c68281eda2ca9ab1df915a2773 from branch mongodb-3.4

        ref: fc0e7abe82..6a31c2118c
        for: 3.4.0

        WT-2831 Skip creating a checkpoint if there have been no changes
        WT-2858 rename wtperf's CONFIG structure
        WT-2895 Reduce the runtime of make check testing with disable long
        WT-2907 Bug in Java ConcurrentCloseTest case
        WT-2917 split wtperf's configuration into per-database and per-run parts
        WT-2920 New eviction statistics
        WT-2931 Configure default in-memory dirty cache usage lower
        WT-2932 Allow applications to selectively ignore cache limit with in-memory configuration
        WT-2933 Fix a race between named snapshots and checkpoints
        WT-2937 test_inmem01 aborts due to stuck cache
        WT-2938 Assembly files should end in .sx, not .S
        WT-2941 Improve test/format to use faster key-generation functions
        WT-2942 verbose strings don't need newline
        WT-2946 dist/s_docs incompatible with OS X Xcode installation
        WT-2948 simplify error handling by making epoch time return never fail
        WT-2950 Inserting multi-megabyte values can cause large in-memory pages
        Branch: master
        https://github.com/mongodb/mongo/commit/448f5b43d249dd090da1d79b82c9ab1b742e4f38

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'} Message: Import wiredtiger: 6a31c2118cce88c68281eda2ca9ab1df915a2773 from branch mongodb-3.4 ref: fc0e7abe82..6a31c2118c for: 3.4.0 WT-2831 Skip creating a checkpoint if there have been no changes WT-2858 rename wtperf's CONFIG structure WT-2895 Reduce the runtime of make check testing with disable long WT-2907 Bug in Java ConcurrentCloseTest case WT-2917 split wtperf's configuration into per-database and per-run parts WT-2920 New eviction statistics WT-2931 Configure default in-memory dirty cache usage lower WT-2932 Allow applications to selectively ignore cache limit with in-memory configuration WT-2933 Fix a race between named snapshots and checkpoints WT-2937 test_inmem01 aborts due to stuck cache WT-2938 Assembly files should end in .sx, not .S WT-2941 Improve test/format to use faster key-generation functions WT-2942 verbose strings don't need newline WT-2946 dist/s_docs incompatible with OS X Xcode installation WT-2948 simplify error handling by making epoch time return never fail WT-2950 Inserting multi-megabyte values can cause large in-memory pages Branch: master https://github.com/mongodb/mongo/commit/448f5b43d249dd090da1d79b82c9ab1b742e4f38
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2933 Fix a race between named snapshots and checkpoints. (#3068)
        Branch: mongodb-3.2
        https://github.com/wiredtiger/wiredtiger/commit/213053e190333eecf91556875317a44ac7f283f6

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2933 Fix a race between named snapshots and checkpoints. (#3068) Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/213053e190333eecf91556875317a44ac7f283f6
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2933 Fix an uninitialized variable warning. (#3077)
        Branch: mongodb-3.2
        https://github.com/wiredtiger/wiredtiger/commit/19445fbfdfb81cefcb4c9f2a67b0fba360aaa515

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2933 Fix an uninitialized variable warning. (#3077) Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/19445fbfdfb81cefcb4c9f2a67b0fba360aaa515
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

        Message: Import wiredtiger: b11ed312cedb905dec49dd2c9c262fabf64d13cd from branch mongodb-3.2

        ref: 9cf2f89d6d..b11ed312ce
        for: 3.2.11

        WT-1592 Dump detailed cache information via statistics
        WT-2403 Enhance random cursor implementation for LSM trees
        WT-2831 Skip creating a checkpoint if there have been no changes
        WT-2858 rename wtperf's CONFIG structure
        WT-2880 Add support for Zstandard compression
        WT-2895 Reduce the runtime of make check testing with disable long
        WT-2904 Fix a bug where the reported checkpoint size could be many times data size
        WT-2907 Bug in Java ConcurrentCloseTest case
        WT-2917 split wtperf's configuration into per-database and per-run parts
        WT-2920 Add statistic tracking application thread cache maintenance time
        WT-2931 Configure default in-memory dirty cache usage lower
        WT-2932 Allow applications to selectively ignore cache limit with in-memory configuration
        WT-2933 Fix a race between named snapshots and checkpoints
        WT-2937 test_inmem01 aborts due to stuck cache
        WT-2938 Assembly files should end in .sx, not .S
        WT-2941 Improve test/format to use faster key-generation functions
        WT-2942 verbose strings don't need newline
        WT-2946 dist/s_docs incompatible with OS X Xcode installation
        WT-2948 simplify error handling by making epoch time return never fail
        WT-2949 Add an option to wtperf to not close connection on shutdown
        WT-2950 Inserting multi-megabyte values can cause large in-memory pages
        WT-2954 Inserting multi-megabyte values can cause large in-memory pages
        WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks
        WT-2956 utility tests -h option is always overridden by the default setup
        WT-2959 Ensure WT_SESSION_IMPL is never used before it's initialized
        WT-2963 Race setting max_entries during eviction
        WT-2965 test_wt2323_join_visibility can hang on OSX
        WT-2974 lint
        WT-2976 Add a statistic tracking how long application threads spend doing I/O
        WT-2977 Csuite LSM Random test can occasionally fail
        WT-2985 Race during checkpoint can cause a core dump
        WT-2987 Fix a bug where opening a cursor on an incomplete table drops core
        WT-2988 __wt_epoch potentially returns garbage values.
        Branch: v3.2
        https://github.com/mongodb/mongo/commit/ebbb4eb0b091fa185b06a060d24b68eb6761ba4a

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'} Message: Import wiredtiger: b11ed312cedb905dec49dd2c9c262fabf64d13cd from branch mongodb-3.2 ref: 9cf2f89d6d..b11ed312ce for: 3.2.11 WT-1592 Dump detailed cache information via statistics WT-2403 Enhance random cursor implementation for LSM trees WT-2831 Skip creating a checkpoint if there have been no changes WT-2858 rename wtperf's CONFIG structure WT-2880 Add support for Zstandard compression WT-2895 Reduce the runtime of make check testing with disable long WT-2904 Fix a bug where the reported checkpoint size could be many times data size WT-2907 Bug in Java ConcurrentCloseTest case WT-2917 split wtperf's configuration into per-database and per-run parts WT-2920 Add statistic tracking application thread cache maintenance time WT-2931 Configure default in-memory dirty cache usage lower WT-2932 Allow applications to selectively ignore cache limit with in-memory configuration WT-2933 Fix a race between named snapshots and checkpoints WT-2937 test_inmem01 aborts due to stuck cache WT-2938 Assembly files should end in .sx, not .S WT-2941 Improve test/format to use faster key-generation functions WT-2942 verbose strings don't need newline WT-2946 dist/s_docs incompatible with OS X Xcode installation WT-2948 simplify error handling by making epoch time return never fail WT-2949 Add an option to wtperf to not close connection on shutdown WT-2950 Inserting multi-megabyte values can cause large in-memory pages WT-2954 Inserting multi-megabyte values can cause large in-memory pages WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks WT-2956 utility tests -h option is always overridden by the default setup WT-2959 Ensure WT_SESSION_IMPL is never used before it's initialized WT-2963 Race setting max_entries during eviction WT-2965 test_wt2323_join_visibility can hang on OSX WT-2974 lint WT-2976 Add a statistic tracking how long application threads spend doing I/O WT-2977 Csuite LSM Random test can occasionally fail WT-2985 Race during checkpoint can cause a core dump WT-2987 Fix a bug where opening a cursor on an incomplete table drops core WT-2988 __wt_epoch potentially returns garbage values. Branch: v3.2 https://github.com/mongodb/mongo/commit/ebbb4eb0b091fa185b06a060d24b68eb6761ba4a

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Days since reply:
              26 weeks ago
              Date of 1st Reply: