Details
Bug
Major - P3
Description
If all of the valid eviction candidates fit into one eviction queue and the only thread doing any eviction is a single eviction server, it can keep trying to fill the empty queue and never make progress. This is one cause of the inmem unit test getting stuck.
Activity
Author:
{u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}Message: WT-2937 Fix a warning. (#3084)
../src/evict/evict_lru.c:1081:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
WT_MIN(max_entries, 1 + __wt_cache_pages_inuse(cache) / 2);
~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN'
^
../src/evict/evict_lru.c:1084:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
WT_MIN(max_entries, 1 + cache->pages_dirty_leaf / 2);
~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN'
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/d6c7a001dded7b000a08d8675592f912ccdbab79
Author:
{u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}Message: WT-2937 Only do an eviction walk after a page has been taken from the current queue. (#3071)
Along with a number of other changes to avoid pathological cases with tiny caches:
Application threads should do urgent eviction if there is only one eviction worker.
Always alternate the "fill queue", so that once a walk has tried to fill one queue, it keeps evicting from the other.
If there are tiny numbers of candidate pages, only allow half in each queue to avoid situations where all candidates are in one queue.
If there is only one candidate page in the whole cache, it had better get queued.
Don't allow the eviction server to sit on pages requiring urgent eviction.
Branch: mongodb-3.4
https://github.com/wiredtiger/wiredtiger/commit/192fbccdcb54ffac071d53ede4d15fc757f4ab54
Author:
{u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}Message: WT-2937 Fix a warning. (#3084)
../src/evict/evict_lru.c:1081:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
WT_MIN(max_entries, 1 + __wt_cache_pages_inuse(cache) / 2);
~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN'
^
../src/evict/evict_lru.c:1084:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
WT_MIN(max_entries, 1 + cache->pages_dirty_leaf / 2);
~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN'
Branch: mongodb-3.4
https://github.com/wiredtiger/wiredtiger/commit/d6c7a001dded7b000a08d8675592f912ccdbab79
Author:
{u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}Message: Import wiredtiger: 6a31c2118cce88c68281eda2ca9ab1df915a2773 from branch mongodb-3.4
ref: fc0e7abe82..6a31c2118c
for: 3.4.0
WT-2831 Skip creating a checkpoint if there have been no changes
WT-2858 rename wtperf's CONFIG structure
WT-2895 Reduce the runtime of make check testing with disable long
WT-2907 Bug in Java ConcurrentCloseTest case
WT-2917 split wtperf's configuration into per-database and per-run parts
WT-2920 New eviction statistics
WT-2931 Configure default in-memory dirty cache usage lower
WT-2932 Allow applications to selectively ignore cache limit with in-memory configuration
WT-2933 Fix a race between named snapshots and checkpoints
WT-2937 test_inmem01 aborts due to stuck cache
WT-2938 Assembly files should end in .sx, not .S
WT-2941 Improve test/format to use faster key-generation functions
WT-2942 verbose strings don't need newline
WT-2946 dist/s_docs incompatible with OS X Xcode installation
WT-2948 simplify error handling by making epoch time return never fail
WT-2950 Inserting multi-megabyte values can cause large in-memory pages
Branch: master
https://github.com/mongodb/mongo/commit/448f5b43d249dd090da1d79b82c9ab1b742e4f38
Author:
{u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}Message: WT-2937 Only do an eviction walk after a page has been taken from the current queue. (#3071)
Along with a number of other changes to avoid pathological cases with tiny caches:
Application threads should do urgent eviction if there is only one eviction worker.
Always alternate the "fill queue", so that once a walk has tried to fill one queue, it keeps evicting from the other.
If there are tiny numbers of candidate pages, only allow half in each queue to avoid situations where all candidates are in one queue.
If there is only one candidate page in the whole cache, it had better get queued.
Don't allow the eviction server to sit on pages requiring urgent eviction.
Branch: mongodb-3.2
https://github.com/wiredtiger/wiredtiger/commit/192fbccdcb54ffac071d53ede4d15fc757f4ab54
Author:
{u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}Message: WT-2937 Fix a warning. (#3084)
../src/evict/evict_lru.c:1081:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
WT_MIN(max_entries, 1 + __wt_cache_pages_inuse(cache) / 2);
~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN'
^
../src/evict/evict_lru.c:1084:29: error: implicit conversion loses integer precision: 'unsigned long long' to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
WT_MIN(max_entries, 1 + cache->pages_dirty_leaf / 2);
~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/include/misc.h:62:42: note: expanded from macro 'WT_MIN'
Branch: mongodb-3.2
https://github.com/wiredtiger/wiredtiger/commit/d6c7a001dded7b000a08d8675592f912ccdbab79
Author:
{u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}Message: Import wiredtiger: b11ed312cedb905dec49dd2c9c262fabf64d13cd from branch mongodb-3.2
ref: 9cf2f89d6d..b11ed312ce
for: 3.2.11
WT-1592 Dump detailed cache information via statistics
WT-2403 Enhance random cursor implementation for LSM trees
WT-2831 Skip creating a checkpoint if there have been no changes
WT-2858 rename wtperf's CONFIG structure
WT-2880 Add support for Zstandard compression
WT-2895 Reduce the runtime of make check testing with disable long
WT-2904 Fix a bug where the reported checkpoint size could be many times data size
WT-2907 Bug in Java ConcurrentCloseTest case
WT-2917 split wtperf's configuration into per-database and per-run parts
WT-2920 Add statistic tracking application thread cache maintenance time
WT-2931 Configure default in-memory dirty cache usage lower
WT-2932 Allow applications to selectively ignore cache limit with in-memory configuration
WT-2933 Fix a race between named snapshots and checkpoints
WT-2937 test_inmem01 aborts due to stuck cache
WT-2938 Assembly files should end in .sx, not .S
WT-2941 Improve test/format to use faster key-generation functions
WT-2942 verbose strings don't need newline
WT-2946 dist/s_docs incompatible with OS X Xcode installation
WT-2948 simplify error handling by making epoch time return never fail
WT-2949 Add an option to wtperf to not close connection on shutdown
WT-2950 Inserting multi-megabyte values can cause large in-memory pages
WT-2954 Inserting multi-megabyte values can cause large in-memory pages
WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks
WT-2956 utility tests -h option is always overridden by the default setup
WT-2959 Ensure WT_SESSION_IMPL is never used before it's initialized
WT-2963 Race setting max_entries during eviction
WT-2965 test_wt2323_join_visibility can hang on OSX
WT-2974 lint
WT-2976 Add a statistic tracking how long application threads spend doing I/O
WT-2977 Csuite LSM Random test can occasionally fail
WT-2985 Race during checkpoint can cause a core dump
WT-2987 Fix a bug where opening a cursor on an incomplete table drops core
WT-2988 __wt_epoch potentially returns garbage values.
Branch: v3.2
https://github.com/mongodb/mongo/commit/ebbb4eb0b091fa185b06a060d24b68eb6761ba4a
Author:
{u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}Message:
WT-2937Only do an eviction walk after a page has been taken from the current queue. (#3071)Along with a number of other changes to avoid pathological cases with tiny caches:
Application threads should do urgent eviction if there is only one eviction worker.
Always alternate the "fill queue", so that once a walk has tried to fill one queue, it keeps evicting from the other.
If there are tiny numbers of candidate pages, only allow half in each queue to avoid situations where all candidates are in one queue.
If there is only one candidate page in the whole cache, it had better get queued.
Don't allow the eviction server to sit on pages requiring urgent eviction.
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/192fbccdcb54ffac071d53ede4d15fc757f4ab54