[SERVER-27347] Only close idle cached cursors on the WiredTiger ident that is busy Created: 08/Dec/16  Updated: 06/Dec/17  Resolved: 16/Jun/17

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 3.2.15, 3.4.6, 3.5.9

Type: Improvement Priority: Major - P3
Reporter: Geert Bosch Assignee: Donald Anderson
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
depends on SERVER-27345 reclaim inactive WT sessions on calli... Closed
depends on WT-3369 WT_CURSOR->uri should always match th... Closed
is depended on by SERVER-27348 Only retry dropping idents on WiredTi... Closed
Duplicate
is duplicated by SERVER-29024 Collection drop slows/stalls down dat... Closed
Related
related to SERVER-30426 dropDatabase very slow due to repeate... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v3.4, v3.2
Sprint: Storage 2017-05-29, Storage 2017-06-19, Storage 2017-07-10
Participants:
Case:

 Description   

Right now, in the WiredTiger integration layer, we generate a new global cursor epoch and close all cursors that are returned to the cache on encountering a WT_BUSY attempting to drop a specific ident. Change closeAllCursors to only close cursors on the idents that require dropping.

This is expected to significantly help users with large numbers of collections.



 Comments   
Comment by Githook User [ 22/Jun/17 ]

Author:

{u'username': u'ddanderson', u'name': u'Don Anderson', u'email': u'dda@mongodb.com'}

Message: SERVER-27347 For table drop, only close relevant URIs.

(cherry picked from commit b9739a9b4bf02790d427a8e4910629f8c378c5fb)
Branch: v3.2
https://github.com/mongodb/mongo/commit/e9ad6a777d762fdae1d54137f7c8fae69784fc2a

Comment by Githook User [ 21/Jun/17 ]

Author:

{u'username': u'ddanderson', u'name': u'Don Anderson', u'email': u'dda@mongodb.com'}

Message: SERVER-27347 For table drop, only close relevant URIs.
Branch: v3.4
https://github.com/mongodb/mongo/commit/5e528ece0ead6ca020ec0be14aa0f080a5265c92

Comment by Donald Anderson [ 16/Jun/17 ]

Note, for this fix to work correctly, a corresponding change to WiredTiger is needed: https://github.com/wiredtiger/wiredtiger/commit/84429199fd94f8a8201c5aa77432a2557d326902

Comment by Githook User [ 16/Jun/17 ]

Author:

{u'username': u'ddanderson', u'name': u'Don Anderson', u'email': u'dda@mongodb.com'}

Message: SERVER-27347 For table drop, only close relevant URIs.
Branch: master
https://github.com/mongodb/mongo/commit/b9739a9b4bf02790d427a8e4910629f8c378c5fb

Comment by Alexander Gorrod [ 06/Jun/17 ]

We have developed a proof of concept change in order to test whether the approach of tuning the MongoDB cursor cache alleviates symptoms in workloads with a lot of collections that drop collections and/or indexes. The following workload was used to simulate conditions that display negative symptoms:

db=`pwd`/data
gb=10
threads=10
ops=1000000
drop_every=10000
tables=100
 
function start {
    killall -9 -w mongod
    rm -rf $db $db.log
    mkdir -p $db
    mongod --dbpath $db --logpath $db.log --storageEngine wiredTiger --nojournal \
        --wiredTigerCacheSizeGB $gb --fork --syncdelay 0
}
 
# generate a collection
function run {
    (
        for t in $(seq $threads); do
            mongo --eval "
	        _srand($t)
		for (var i=0; i<$ops; i++) {
	            cn=Math.floor(_rand() * $tables)
                    c = db['c' + cn]
		    if (i % $drop_every == 0)
			    c.drop()
		    else
			    c.insert({a:i, b:i, c:i})
	        }
            " &
        done
        wait
    )
}
 
start      # start mongod
run

The prototype change passes most of our automated tests, the exception is that the prototype causes validate_tests in the dbtest_WT suite to fail, because the changes mean we are no longer closing all cursors the first time WT_SESSION::verify returns EBUSY. I haven't looked closely enough to know whether there is a way to make that test pass again without going back to having validate potentially interrupt unrelated operations by closing all cursors.

Generated at Thu Feb 08 04:14:54 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.