Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2937

test_inmem01 aborts due to stuck cache

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.9.0, 3.2.11, 3.4.0-rc1
    • Labels:
      None
    • # Replies:
      8
    • Last comment by Customer:
      true

      Description

      If all of the valid eviction candidates fit into one eviction queue and the only thread doing any eviction is a single eviction server, it can keep trying to fill the empty queue and never make progress. This is one cause of the inmem unit test getting stuck.

        Activity

        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2937 Only do an eviction walk after a page has been taken from the current queue. (#3071)

        Along with a number of other changes to avoid pathological cases with tiny caches:

        Application threads should do urgent eviction if there is only one eviction worker.
        Always alternate the "fill queue", so that once a walk has tried to fill one queue, it keeps evicting from the other.

        If there are tiny numbers of candidate pages, only allow half in each queue to avoid situations where all candidates are in one queue.

        If there is only one candidate page in the whole cache, it had better get queued.

        Don't allow the eviction server to sit on pages requiring urgent eviction.
        Branch: develop
        https://github.com/wiredtiger/wiredtiger/commit/192fbccdcb54ffac071d53ede4d15fc757f4ab54

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2937 Only do an eviction walk after a page has been taken from the current queue. (#3071) Along with a number of other changes to avoid pathological cases with tiny caches: Application threads should do urgent eviction if there is only one eviction worker. Always alternate the "fill queue", so that once a walk has tried to fill one queue, it keeps evicting from the other. If there are tiny numbers of candidate pages, only allow half in each queue to avoid situations where all candidates are in one queue. If there is only one candidate page in the whole cache, it had better get queued. Don't allow the eviction server to sit on pages requiring urgent eviction. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/192fbccdcb54ffac071d53ede4d15fc757f4ab54
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2937 Fix a warning. (#3084)

        ../src/evict/evict_lru.c:1081:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
        WT_MIN(max_entries, 1 + __wt_cache_pages_inuse(cache) / 2);
        ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        ../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN'
        ^
        ../src/evict/evict_lru.c:1084:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
        WT_MIN(max_entries, 1 + cache->pages_dirty_leaf / 2);
        ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
        ../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN'
        Branch: develop
        https://github.com/wiredtiger/wiredtiger/commit/d6c7a001dded7b000a08d8675592f912ccdbab79

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2937 Fix a warning. (#3084) ../src/evict/evict_lru.c:1081:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32] WT_MIN(max_entries, 1 + __wt_cache_pages_inuse(cache) / 2); ~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN' ^ ../src/evict/evict_lru.c:1084:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32] WT_MIN(max_entries, 1 + cache->pages_dirty_leaf / 2); ~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN' Branch: develop https://github.com/wiredtiger/wiredtiger/commit/d6c7a001dded7b000a08d8675592f912ccdbab79
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2937 Only do an eviction walk after a page has been taken from the current queue. (#3071)

        Along with a number of other changes to avoid pathological cases with tiny caches:

        Application threads should do urgent eviction if there is only one eviction worker.
        Always alternate the "fill queue", so that once a walk has tried to fill one queue, it keeps evicting from the other.

        If there are tiny numbers of candidate pages, only allow half in each queue to avoid situations where all candidates are in one queue.

        If there is only one candidate page in the whole cache, it had better get queued.

        Don't allow the eviction server to sit on pages requiring urgent eviction.
        Branch: mongodb-3.4
        https://github.com/wiredtiger/wiredtiger/commit/192fbccdcb54ffac071d53ede4d15fc757f4ab54

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2937 Only do an eviction walk after a page has been taken from the current queue. (#3071) Along with a number of other changes to avoid pathological cases with tiny caches: Application threads should do urgent eviction if there is only one eviction worker. Always alternate the "fill queue", so that once a walk has tried to fill one queue, it keeps evicting from the other. If there are tiny numbers of candidate pages, only allow half in each queue to avoid situations where all candidates are in one queue. If there is only one candidate page in the whole cache, it had better get queued. Don't allow the eviction server to sit on pages requiring urgent eviction. Branch: mongodb-3.4 https://github.com/wiredtiger/wiredtiger/commit/192fbccdcb54ffac071d53ede4d15fc757f4ab54
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2937 Fix a warning. (#3084)

        ../src/evict/evict_lru.c:1081:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
        WT_MIN(max_entries, 1 + __wt_cache_pages_inuse(cache) / 2);
        ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        ../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN'
        ^
        ../src/evict/evict_lru.c:1084:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
        WT_MIN(max_entries, 1 + cache->pages_dirty_leaf / 2);
        ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
        ../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN'
        Branch: mongodb-3.4
        https://github.com/wiredtiger/wiredtiger/commit/d6c7a001dded7b000a08d8675592f912ccdbab79

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2937 Fix a warning. (#3084) ../src/evict/evict_lru.c:1081:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32] WT_MIN(max_entries, 1 + __wt_cache_pages_inuse(cache) / 2); ~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN' ^ ../src/evict/evict_lru.c:1084:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32] WT_MIN(max_entries, 1 + cache->pages_dirty_leaf / 2); ~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN' Branch: mongodb-3.4 https://github.com/wiredtiger/wiredtiger/commit/d6c7a001dded7b000a08d8675592f912ccdbab79
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

        Message: Import wiredtiger: 6a31c2118cce88c68281eda2ca9ab1df915a2773 from branch mongodb-3.4

        ref: fc0e7abe82..6a31c2118c
        for: 3.4.0

        WT-2831 Skip creating a checkpoint if there have been no changes
        WT-2858 rename wtperf's CONFIG structure
        WT-2895 Reduce the runtime of make check testing with disable long
        WT-2907 Bug in Java ConcurrentCloseTest case
        WT-2917 split wtperf's configuration into per-database and per-run parts
        WT-2920 New eviction statistics
        WT-2931 Configure default in-memory dirty cache usage lower
        WT-2932 Allow applications to selectively ignore cache limit with in-memory configuration
        WT-2933 Fix a race between named snapshots and checkpoints
        WT-2937 test_inmem01 aborts due to stuck cache
        WT-2938 Assembly files should end in .sx, not .S
        WT-2941 Improve test/format to use faster key-generation functions
        WT-2942 verbose strings don't need newline
        WT-2946 dist/s_docs incompatible with OS X Xcode installation
        WT-2948 simplify error handling by making epoch time return never fail
        WT-2950 Inserting multi-megabyte values can cause large in-memory pages
        Branch: master
        https://github.com/mongodb/mongo/commit/448f5b43d249dd090da1d79b82c9ab1b742e4f38

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'} Message: Import wiredtiger: 6a31c2118cce88c68281eda2ca9ab1df915a2773 from branch mongodb-3.4 ref: fc0e7abe82..6a31c2118c for: 3.4.0 WT-2831 Skip creating a checkpoint if there have been no changes WT-2858 rename wtperf's CONFIG structure WT-2895 Reduce the runtime of make check testing with disable long WT-2907 Bug in Java ConcurrentCloseTest case WT-2917 split wtperf's configuration into per-database and per-run parts WT-2920 New eviction statistics WT-2931 Configure default in-memory dirty cache usage lower WT-2932 Allow applications to selectively ignore cache limit with in-memory configuration WT-2933 Fix a race between named snapshots and checkpoints WT-2937 test_inmem01 aborts due to stuck cache WT-2938 Assembly files should end in .sx, not .S WT-2941 Improve test/format to use faster key-generation functions WT-2942 verbose strings don't need newline WT-2946 dist/s_docs incompatible with OS X Xcode installation WT-2948 simplify error handling by making epoch time return never fail WT-2950 Inserting multi-megabyte values can cause large in-memory pages Branch: master https://github.com/mongodb/mongo/commit/448f5b43d249dd090da1d79b82c9ab1b742e4f38
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2937 Only do an eviction walk after a page has been taken from the current queue. (#3071)

        Along with a number of other changes to avoid pathological cases with tiny caches:

        Application threads should do urgent eviction if there is only one eviction worker.
        Always alternate the "fill queue", so that once a walk has tried to fill one queue, it keeps evicting from the other.

        If there are tiny numbers of candidate pages, only allow half in each queue to avoid situations where all candidates are in one queue.

        If there is only one candidate page in the whole cache, it had better get queued.

        Don't allow the eviction server to sit on pages requiring urgent eviction.
        Branch: mongodb-3.2
        https://github.com/wiredtiger/wiredtiger/commit/192fbccdcb54ffac071d53ede4d15fc757f4ab54

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2937 Only do an eviction walk after a page has been taken from the current queue. (#3071) Along with a number of other changes to avoid pathological cases with tiny caches: Application threads should do urgent eviction if there is only one eviction worker. Always alternate the "fill queue", so that once a walk has tried to fill one queue, it keeps evicting from the other. If there are tiny numbers of candidate pages, only allow half in each queue to avoid situations where all candidates are in one queue. If there is only one candidate page in the whole cache, it had better get queued. Don't allow the eviction server to sit on pages requiring urgent eviction. Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/192fbccdcb54ffac071d53ede4d15fc757f4ab54
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: WT-2937 Fix a warning. (#3084)

        ../src/evict/evict_lru.c:1081:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
        WT_MIN(max_entries, 1 + __wt_cache_pages_inuse(cache) / 2);
        ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        ../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN'
        ^
        ../src/evict/evict_lru.c:1084:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
        WT_MIN(max_entries, 1 + cache->pages_dirty_leaf / 2);
        ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
        ../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN'
        Branch: mongodb-3.2
        https://github.com/wiredtiger/wiredtiger/commit/d6c7a001dded7b000a08d8675592f912ccdbab79

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2937 Fix a warning. (#3084) ../src/evict/evict_lru.c:1081:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32] WT_MIN(max_entries, 1 + __wt_cache_pages_inuse(cache) / 2); ~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN' ^ ../src/evict/evict_lru.c:1084:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32] WT_MIN(max_entries, 1 + cache->pages_dirty_leaf / 2); ~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN' Branch: mongodb-3.2 https://github.com/wiredtiger/wiredtiger/commit/d6c7a001dded7b000a08d8675592f912ccdbab79
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

        Message: Import wiredtiger: b11ed312cedb905dec49dd2c9c262fabf64d13cd from branch mongodb-3.2

        ref: 9cf2f89d6d..b11ed312ce
        for: 3.2.11

        WT-1592 Dump detailed cache information via statistics
        WT-2403 Enhance random cursor implementation for LSM trees
        WT-2831 Skip creating a checkpoint if there have been no changes
        WT-2858 rename wtperf's CONFIG structure
        WT-2880 Add support for Zstandard compression
        WT-2895 Reduce the runtime of make check testing with disable long
        WT-2904 Fix a bug where the reported checkpoint size could be many times data size
        WT-2907 Bug in Java ConcurrentCloseTest case
        WT-2917 split wtperf's configuration into per-database and per-run parts
        WT-2920 Add statistic tracking application thread cache maintenance time
        WT-2931 Configure default in-memory dirty cache usage lower
        WT-2932 Allow applications to selectively ignore cache limit with in-memory configuration
        WT-2933 Fix a race between named snapshots and checkpoints
        WT-2937 test_inmem01 aborts due to stuck cache
        WT-2938 Assembly files should end in .sx, not .S
        WT-2941 Improve test/format to use faster key-generation functions
        WT-2942 verbose strings don't need newline
        WT-2946 dist/s_docs incompatible with OS X Xcode installation
        WT-2948 simplify error handling by making epoch time return never fail
        WT-2949 Add an option to wtperf to not close connection on shutdown
        WT-2950 Inserting multi-megabyte values can cause large in-memory pages
        WT-2954 Inserting multi-megabyte values can cause large in-memory pages
        WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks
        WT-2956 utility tests -h option is always overridden by the default setup
        WT-2959 Ensure WT_SESSION_IMPL is never used before it's initialized
        WT-2963 Race setting max_entries during eviction
        WT-2965 test_wt2323_join_visibility can hang on OSX
        WT-2974 lint
        WT-2976 Add a statistic tracking how long application threads spend doing I/O
        WT-2977 Csuite LSM Random test can occasionally fail
        WT-2985 Race during checkpoint can cause a core dump
        WT-2987 Fix a bug where opening a cursor on an incomplete table drops core
        WT-2988 __wt_epoch potentially returns garbage values.
        Branch: v3.2
        https://github.com/mongodb/mongo/commit/ebbb4eb0b091fa185b06a060d24b68eb6761ba4a

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'} Message: Import wiredtiger: b11ed312cedb905dec49dd2c9c262fabf64d13cd from branch mongodb-3.2 ref: 9cf2f89d6d..b11ed312ce for: 3.2.11 WT-1592 Dump detailed cache information via statistics WT-2403 Enhance random cursor implementation for LSM trees WT-2831 Skip creating a checkpoint if there have been no changes WT-2858 rename wtperf's CONFIG structure WT-2880 Add support for Zstandard compression WT-2895 Reduce the runtime of make check testing with disable long WT-2904 Fix a bug where the reported checkpoint size could be many times data size WT-2907 Bug in Java ConcurrentCloseTest case WT-2917 split wtperf's configuration into per-database and per-run parts WT-2920 Add statistic tracking application thread cache maintenance time WT-2931 Configure default in-memory dirty cache usage lower WT-2932 Allow applications to selectively ignore cache limit with in-memory configuration WT-2933 Fix a race between named snapshots and checkpoints WT-2937 test_inmem01 aborts due to stuck cache WT-2938 Assembly files should end in .sx, not .S WT-2941 Improve test/format to use faster key-generation functions WT-2942 verbose strings don't need newline WT-2946 dist/s_docs incompatible with OS X Xcode installation WT-2948 simplify error handling by making epoch time return never fail WT-2949 Add an option to wtperf to not close connection on shutdown WT-2950 Inserting multi-megabyte values can cause large in-memory pages WT-2954 Inserting multi-megabyte values can cause large in-memory pages WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks WT-2956 utility tests -h option is always overridden by the default setup WT-2959 Ensure WT_SESSION_IMPL is never used before it's initialized WT-2963 Race setting max_entries during eviction WT-2965 test_wt2323_join_visibility can hang on OSX WT-2974 lint WT-2976 Add a statistic tracking how long application threads spend doing I/O WT-2977 Csuite LSM Random test can occasionally fail WT-2985 Race during checkpoint can cause a core dump WT-2987 Fix a bug where opening a cursor on an incomplete table drops core WT-2988 __wt_epoch potentially returns garbage values. Branch: v3.2 https://github.com/mongodb/mongo/commit/ebbb4eb0b091fa185b06a060d24b68eb6761ba4a

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Days since reply:
              30 weeks, 2 days ago
              Date of 1st Reply: