Details
Description
In the past, WiredTiger has kept metadata pinned in cache. That prevents any eviction from the metadata table, including cleaning up of tombstones. As a result, workloads that create and drop many tables can grow the metadata table in cache without bound.
Once the metadata table grows larger than the cache size, other operations are blocked.
Issue Links
- is depended on by
-
SERVER-22964
IX GlobalLock being held while wating for wt cache eviction
-
- Closed
-
-
SERVER-23140
WiredTiger changes for MongoDB 3.3.4
-
- Closed
-
Activity
Author:
{u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}Message: Import wiredtiger-wiredtiger-2.7.0-1181-g43e885a.tar.gz from wiredtiger branch mongodb-3.2
ref: 5cdd3e3..43e885a
SERVER-22676 WiredTiger fails to open databases created by 3.0.0 or 3.0.1
SERVER-22773 gle_shell_server5441.js fails on ppc64le
SERVER-22784 Coverity analysis defect 77722: Unused value
SERVER-22831 Low query rate with heavy cache pressure and an idle collection
SERVER-23040 Coverity analysis defect 98151: Dereference after null check
SERVER-23203 WiredTiger changes for MongoDB 3.2.5
WT-2107 Add example code including an event handler
WT-2123 Don't clear allocated memory if not required
WT-2173 test/format cache stuck full
WT-2264 Checkpoints cannot keep up with inserts
WT-2280 Add CRC32 Optimized code for PPC64LE
WT-2295 WT_SESSION.create does a full-scan of the main table
WT-2318 Configurable thread wake up time
WT-2322 Join cursor with isolation read-uncommitted may give different results with Bloom filters
WT-2345 Evicting tiny pages creates small pages on disk
WT-2346 Don't hold schema lock during checkpoint I/O
WT-2349 Add ability to open databases read-only
WT-2361 column-store starting record number error
WT-2366 Extend wtperf to support updates that grow the record size
WT-2367 WT_CURSOR.next out-of-order returns failure
WT-2374 read error on index file
WT-2375 Need tests for collators
WT-2376 Modules should compile without including wt_internal.h header file
WT-2381 dump utility discards table config
WT-2382 Problem with custom collator for 'u' format with join cursor
WT-2384 lt, le conditions for ordering cursor in join cursor
WT-2387 Fix cursor random unit test on Windows
WT-2390 OS X build is broken
WT-2391 De-prioritize eviction from indexes
WT-2393 Unnecessary error handling labels.
WT-2394 Long Unit Test for test_compact02 failed.
WT-2395 Recovery failure with an LSM tree
WT-2396 Jenkins Spinlock GCC task Hung
WT-2397 Cursor traversal from end of the tree skips records.
WT-2399 Add test case that verifies cursor traversal
WT-2404 Add streaming pack/unpack methods to the extension API
WT-2405 test utility error handling.
WT-2406 cursor_order lint, minor bug fixes
WT-2407 recovery lint
WT-2409 Minor Perf Regression in LSM
WT-2410 Casting function pointers to different types
WT-2411 LSM drop hang
WT-2412 Truncate error tag is incorrect
WT-2414 Avoid extractor calls for ordering cursor in join cursor
WT-2417 Windows Jenkins task is failing
WT-2418 test_rebalance failing with EBUSY
WT-2419 Tests fail to compile on windows due to new util functions
WT-2420 LSM failed to read bytes
WT-2421 test_bloom ret assigned but not used
WT-2422 multiple definitions of custom die function
WT-2423 Session reference count could be wrong if adding the handle fails
WT-2425 evict-btree read through performance drop
WT-2426 Deadlock caused by recent changes to checkpoint handle locking
WT-2427 wtperf should work with builtin compression
WT-2428 Make statistics logging compatible with MongoDB
WT-2429 Add a statistic that tracks aggressive mode in eviction
WT-2430 statistic for join cursor
WT-2431 Join statistics documentation needed
WT-2432 Understand eviction impact on LSM and readonly workload
WT-2433 Allow read-only databases to log statistics
WT-2434 Race between force-drop and sweep
WT-2435 __wt_evict_file_exclusive_on/off cleanups
WT-2436 lt, le conditions for ref cursor with "strategy=bloom" in join cursor
WT-2437 Test suite failures on Windows
WT-2438 Extend WiredTiger stat declarations to help timeseries tool
WT-2440 vpmsum_crc32: Per the PPC64 ABI, v20-v31 are non-volatile register
WT-2443 Getting statistic for all indexes used in join cursor
WT-2444 broken flag test in wtperf, whitespace
WT-2447 join cursor reads main table
WT-2448 Add no_scale flag to relevant statistics
WT-2449 configure should check for a 64-bit build
WT-2451 Allow eviction of metadata
WT-2454 checkpoint_sync=false does not prevent flushes/sync to disk.
WT-2456 Update Power8 CRC32 Code
WT-2457 Dropping an LSM table can fail with EBUSY when no user ops are active
WT-2459 Allow Configure scripts to provide the --tag option for libtool when compiling on PPC
WT-2460 checkpoint failing with WT_ROLLBACK
WT-2461 sweep01 failing
WT-2463 Test that measures idle CPU usage fails under valgrind
WT-2464 Valgrind errors.
WT-2465 Coverity 1352899: Dereference before null check
WT-2466 Coverity 1352893 Buffer not null terminated
WT-2467 Coverity 1352894: Logically dead code
WT-2468 Coverity 1352896: Explicit null dereferenced
WT-2469 Coverity 1352897: Integer overflowed argument
WT-2470 Coverity 1352898: Resource leak
WT-2471 review WiredTiger "int" printf formats
WT-2473 MSVC doesn't support PRId64
WT-2475 Have reconf script remove cached configure results
WT-2476 btree->evict_lock is being accessed after being destroyed
WT-2477 Missing define in Windows wiredtiger_config.h
WT-2478 Valgrind test failures
WT-2481 Recent changes affect LSM performance
WT-2482 Coverity 1353015, 1353016, out-of-bounds access
WT-2483 readonly02 periodically fails
WT-2484 Coverity 1345809: unchecked return value
WT-2485 Test/format failure with Floating point exception
WT-2487 Release memory in manydbs test
WT-2489 warnings from /test/manydbs
WT-2490 search_near() returns wrong key for column-store
WT-2492 Windows test_config04.test_config04.test_invalid_config crashes
WT-2493 verbose lsm_manager unsupported
WT-2494 review calls to __wt_free, plus minor bug in an error path.
WT-2495 Missing memory initialization leads to crash on Windows
WT-2496 test/format unable to read root page
WT-2497 test/format saves copy of backup
WT-2498 LSM tree drop hangs when a user cursor is open
WT-2499 LSM shutdown race causes segfault
WT-2501 Dropping a just opened LSM tree isn't safe
WT-2502 memory leak in locking handles for checkpoint
WT-2503 build warning in lsm_tree.c
WT-2506 Using an uninitialised value
Branch: v3.2
https://github.com/mongodb/mongo/commit/224299a6712196228b65831b6c39498caf8669d2
Author:
{u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexg@wiredtiger.com'}Message: Import wiredtiger-wiredtiger-2.7.0-1122-g9cf8eb2.tar.gz from wiredtiger branch mongodb-3.4
ref: 444981a..9cf8eb2
SERVER-23040 Coverity analysis defect 98151: Dereference after null check
WT-2123 Don't clear allocated memory if not required
WT-2318 Configurable thread wake up time
WT-2322 Join cursor with isolation read-uncommitted may give different results with Bloom filters
WT-2345 Evicting tiny pages creates small pages on disk
WT-2375 Add tests for collators
WT-2381 Dump utility discards table config
WT-2384 lt, le conditions for ordering cursor in join cursor
WT-2391 De-prioritize eviction from indexes
WT-2404 Add streaming pack/unpack methods to the extension API
WT-2414 Avoid extractor calls for ordering cursor in join cursor
WT-2418 Rebalance operation failing with EBUSY
WT-2426 Deadlock caused by recent changes to checkpoint handle locking
WT-2431 Join statistics documentation needed
WT-2435 __wt_evict_file_exclusive_on/off cleanups
WT-2436 lt, le conditions for ref cursor with "strategy=bloom" in join cursor
WT-2443 Getting statistic for all indexes used in join cursor
WT-2444 Broken flag test in wtperf, whitespace
WT-2447 Join cursor reads main table
WT-2448 Add no_scale flag to relevant statistics
WT-2449 Configure should check for a 64-bit build
WT-2451 Allow eviction of metadata
WT-2454 checkpoint_sync=false does not prevent flushes/sync to disk.
WT-2456 Update Power8 CRC32 Code
WT-2457 Dropping an LSM table can fail with EBUSY when no user ops are active
WT-2459 Allow Configure scripts to provide the --tag option for libtool when compiling on PPC
WT-2460 Checkpoint failing with WT_ROLLBACK
WT-2461 Python sweep01 test failing
WT-2463 Test that measures idle CPU usage fails under valgrind
WT-2464 Valgrind errors.
WT-2465 Coverity 1352899: Dereference before null check
WT-2466 Coverity 1352893 Buffer not null terminated
WT-2467 Coverity 1352894: Logically dead code
WT-2468 Coverity 1352896: Explicit null dereferenced
WT-2469 Coverity 1352897: Integer overflowed argument
WT-2470 Coverity 1352898: Resource leak
WT-2471 Review WiredTiger "int" printf formats
WT-2473 MSVC doesn't support PRId64
WT-2475 Have reconf script remove cached configure results
WT-2476 btree->evict_lock is being accessed after being destroyed
WT-2477 Missing define in Windows wiredtiger_config.h
WT-2478 Valgrind test failures
WT-2481 Recent changes affect LSM performance
WT-2482 Coverity 1353015, 1353016, out-of-bounds access
WT-2483 readonly02 periodically fails
WT-2484 Coverity 1345809: unchecked return value
WT-2485 Test/format failure with Floating point exception
WT-2487 Release memory in manydbs test
WT-2489 Fix compiler warnings from /test/manydbs
WT-2490 search_near() returns wrong key for column-store
WT-2492 Windows test_config04.test_config04.test_invalid_config crashes
WT-2493 Verbose lsm_manager unsupported
WT-2494 Review calls to __wt_free, plus minor bug in an error path.
WT-2495 Missing memory initialization leads to crash on Windows
WT-2496 Testing revealed error unable to read root page
WT-2497 Enhance test/format to save a copy of backup
WT-2498 LSM tree drop hangs when a user cursor is open
WT-2499 LSM shutdown race causes segfault
WT-2501 Dropping a just opened LSM tree isn't safe
WT-2502 Memory leak in locking handles for checkpoint
WT-2503 Build warning in lsm_tree.c
WT-2506 Using an uninitialised value
Branch: master
https://github.com/mongodb/mongo/commit/945022433fa60895a6bf412414a89da5a8c14e8c
Author:
{u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}Message: Import wiredtiger-wiredtiger-2.7.0-1181-g43e885a.tar.gz from wiredtiger branch mongodb-3.2
ref: 5cdd3e3..43e885a
SERVER-22676 WiredTiger fails to open databases created by 3.0.0 or 3.0.1
SERVER-22773 gle_shell_server5441.js fails on ppc64le
SERVER-22784 Coverity analysis defect 77722: Unused value
SERVER-22831 Low query rate with heavy cache pressure and an idle collection
SERVER-23040 Coverity analysis defect 98151: Dereference after null check
SERVER-23203 WiredTiger changes for MongoDB 3.2.5
WT-2107 Add example code including an event handler
WT-2123 Don't clear allocated memory if not required
WT-2173 test/format cache stuck full
WT-2264 Checkpoints cannot keep up with inserts
WT-2280 Add CRC32 Optimized code for PPC64LE
WT-2295 WT_SESSION.create does a full-scan of the main table
WT-2318 Configurable thread wake up time
WT-2322 Join cursor with isolation read-uncommitted may give different results with Bloom filters
WT-2345 Evicting tiny pages creates small pages on disk
WT-2346 Don't hold schema lock during checkpoint I/O
WT-2349 Add ability to open databases read-only
WT-2361 column-store starting record number error
WT-2366 Extend wtperf to support updates that grow the record size
WT-2367 WT_CURSOR.next out-of-order returns failure
WT-2374 read error on index file
WT-2375 Need tests for collators
WT-2376 Modules should compile without including wt_internal.h header file
WT-2381 dump utility discards table config
WT-2382 Problem with custom collator for 'u' format with join cursor
WT-2384 lt, le conditions for ordering cursor in join cursor
WT-2387 Fix cursor random unit test on Windows
WT-2390 OS X build is broken
WT-2391 De-prioritize eviction from indexes
WT-2393 Unnecessary error handling labels.
WT-2394 Long Unit Test for test_compact02 failed.
WT-2395 Recovery failure with an LSM tree
WT-2396 Jenkins Spinlock GCC task Hung
WT-2397 Cursor traversal from end of the tree skips records.
WT-2399 Add test case that verifies cursor traversal
WT-2404 Add streaming pack/unpack methods to the extension API
WT-2405 test utility error handling.
WT-2406 cursor_order lint, minor bug fixes
WT-2407 recovery lint
WT-2409 Minor Perf Regression in LSM
WT-2410 Casting function pointers to different types
WT-2411 LSM drop hang
WT-2412 Truncate error tag is incorrect
WT-2414 Avoid extractor calls for ordering cursor in join cursor
WT-2417 Windows Jenkins task is failing
WT-2418 test_rebalance failing with EBUSY
WT-2419 Tests fail to compile on windows due to new util functions
WT-2420 LSM failed to read bytes
WT-2421 test_bloom ret assigned but not used
WT-2422 multiple definitions of custom die function
WT-2423 Session reference count could be wrong if adding the handle fails
WT-2425 evict-btree read through performance drop
WT-2426 Deadlock caused by recent changes to checkpoint handle locking
WT-2427 wtperf should work with builtin compression
WT-2428 Make statistics logging compatible with MongoDB
WT-2429 Add a statistic that tracks aggressive mode in eviction
WT-2430 statistic for join cursor
WT-2431 Join statistics documentation needed
WT-2432 Understand eviction impact on LSM and readonly workload
WT-2433 Allow read-only databases to log statistics
WT-2434 Race between force-drop and sweep
WT-2435 __wt_evict_file_exclusive_on/off cleanups
WT-2436 lt, le conditions for ref cursor with "strategy=bloom" in join cursor
WT-2437 Test suite failures on Windows
WT-2438 Extend WiredTiger stat declarations to help timeseries tool
WT-2440 vpmsum_crc32: Per the PPC64 ABI, v20-v31 are non-volatile register
WT-2443 Getting statistic for all indexes used in join cursor
WT-2444 broken flag test in wtperf, whitespace
WT-2447 join cursor reads main table
WT-2448 Add no_scale flag to relevant statistics
WT-2449 configure should check for a 64-bit build
WT-2451 Allow eviction of metadata
WT-2454 checkpoint_sync=false does not prevent flushes/sync to disk.
WT-2456 Update Power8 CRC32 Code
WT-2457 Dropping an LSM table can fail with EBUSY when no user ops are active
WT-2459 Allow Configure scripts to provide the --tag option for libtool when compiling on PPC
WT-2460 checkpoint failing with WT_ROLLBACK
WT-2461 sweep01 failing
WT-2463 Test that measures idle CPU usage fails under valgrind
WT-2464 Valgrind errors.
WT-2465 Coverity 1352899: Dereference before null check
WT-2466 Coverity 1352893 Buffer not null terminated
WT-2467 Coverity 1352894: Logically dead code
WT-2468 Coverity 1352896: Explicit null dereferenced
WT-2469 Coverity 1352897: Integer overflowed argument
WT-2470 Coverity 1352898: Resource leak
WT-2471 review WiredTiger "int" printf formats
WT-2473 MSVC doesn't support PRId64
WT-2475 Have reconf script remove cached configure results
WT-2476 btree->evict_lock is being accessed after being destroyed
WT-2477 Missing define in Windows wiredtiger_config.h
WT-2478 Valgrind test failures
WT-2481 Recent changes affect LSM performance
WT-2482 Coverity 1353015, 1353016, out-of-bounds access
WT-2483 readonly02 periodically fails
WT-2484 Coverity 1345809: unchecked return value
WT-2485 Test/format failure with Floating point exception
WT-2487 Release memory in manydbs test
WT-2489 warnings from /test/manydbs
WT-2490 search_near() returns wrong key for column-store
WT-2492 Windows test_config04.test_config04.test_invalid_config crashes
WT-2493 verbose lsm_manager unsupported
WT-2494 review calls to __wt_free, plus minor bug in an error path.
WT-2495 Missing memory initialization leads to crash on Windows
WT-2496 test/format unable to read root page
WT-2497 test/format saves copy of backup
WT-2498 LSM tree drop hangs when a user cursor is open
WT-2499 LSM shutdown race causes segfault
WT-2501 Dropping a just opened LSM tree isn't safe
WT-2502 memory leak in locking handles for checkpoint
WT-2503 build warning in lsm_tree.c
WT-2506 Using an uninitialised value
Branch: v3.2
https://github.com/mongodb/mongo/commit/224299a6712196228b65831b6c39498caf8669d2
Author:
{u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}Message: Merge pull request #2556 from wiredtiger/wt-2451
(cherry picked from commit c4f9f62)
WT-2451 Allow eviction of WiredTiger metadata.
Branch: mongodb-3.0
https://github.com/wiredtiger/wiredtiger/commit/48b1f5f607f804aad09c36978e5709e7c4ff16f6
Author:
{u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}Message: Import wiredtiger-wiredtiger-mongodb-3.0.9-9-gf6286c2.tar.gz from wiredtiger branch mongodb-3.0
ref: 3dbc6c6..f6286c2
SERVER-22831 Low query rate with heavy cache pressure and an idle collection
SERVER-23457 WiredTiger changes for MongoDB 3.0.12
WT-2157 test/format corrupted cell failure
WT-2361 column-store starting record number error
WT-2451 Allow eviction of metadata
Branch: v3.0
https://github.com/mongodb/mongo/commit/d0324043bc99a713961e1fca0ffc8ea4b124d959
I suspect
WT-2455is another instance of this although that user reports it taking a very long time (30 minutes) rather than hanging forever.