[SERVER-27347] Only close idle cached cursors on the WiredTiger ident that is busy Created: 08/Dec/16 Updated: 06/Dec/17 Resolved: 16/Jun/17 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | 3.2.15, 3.4.6, 3.5.9 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Geert Bosch | Assignee: | Donald Anderson |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||||||
| Backport Requested: |
v3.4, v3.2
|
||||||||||||||||||||||||||||||||||||
| Sprint: | Storage 2017-05-29, Storage 2017-06-19, Storage 2017-07-10 | ||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||||||||||||||
| Description |
|
Right now, in the WiredTiger integration layer, we generate a new global cursor epoch and close all cursors that are returned to the cache on encountering a WT_BUSY attempting to drop a specific ident. Change closeAllCursors to only close cursors on the idents that require dropping. This is expected to significantly help users with large numbers of collections. |
| Comments |
| Comment by Githook User [ 22/Jun/17 ] | |||||||||||||||||||||||||||||||||||||
|
Author: {u'username': u'ddanderson', u'name': u'Don Anderson', u'email': u'dda@mongodb.com'}Message: (cherry picked from commit b9739a9b4bf02790d427a8e4910629f8c378c5fb) | |||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 21/Jun/17 ] | |||||||||||||||||||||||||||||||||||||
|
Author: {u'username': u'ddanderson', u'name': u'Don Anderson', u'email': u'dda@mongodb.com'}Message: | |||||||||||||||||||||||||||||||||||||
| Comment by Donald Anderson [ 16/Jun/17 ] | |||||||||||||||||||||||||||||||||||||
|
Note, for this fix to work correctly, a corresponding change to WiredTiger is needed: https://github.com/wiredtiger/wiredtiger/commit/84429199fd94f8a8201c5aa77432a2557d326902 | |||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 16/Jun/17 ] | |||||||||||||||||||||||||||||||||||||
|
Author: {u'username': u'ddanderson', u'name': u'Don Anderson', u'email': u'dda@mongodb.com'}Message: | |||||||||||||||||||||||||||||||||||||
| Comment by Alexander Gorrod [ 06/Jun/17 ] | |||||||||||||||||||||||||||||||||||||
|
We have developed a proof of concept change in order to test whether the approach of tuning the MongoDB cursor cache alleviates symptoms in workloads with a lot of collections that drop collections and/or indexes. The following workload was used to simulate conditions that display negative symptoms:
The prototype change passes most of our automated tests, the exception is that the prototype causes validate_tests in the dbtest_WT suite to fail, because the changes mean we are no longer closing all cursors the first time WT_SESSION::verify returns EBUSY. I haven't looked closely enough to know whether there is a way to make that test pass again without going back to having validate potentially interrupt unrelated operations by closing all cursors. |