[SERVER-38779] Build a mechanism to periodically cleanup old WT sessions from session cache Created: 27/Dec/18 Updated: 29/Oct/23 Resolved: 16/Jan/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | 3.6.11, 4.0.6, 4.1.7 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Sulabh Mahajan | Assignee: | Sulabh Mahajan |
| Resolution: | Fixed | Votes: | 4 |
| Labels: | RF | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||
| Backport Requested: |
v4.0, v3.6
|
||||||||||||||||||||
| Sprint: | Storage Engines 2018-12-31, Storage Engines 2019-01-14, Storage Engines 2019-01-28 | ||||||||||||||||||||
| Participants: | |||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||
| Story Points: | 8 | ||||||||||||||||||||
| Description |
|
The way session cache is maintained, idle sessions keep accumulating in the session cache. If the workload doesn't use all the idle sessions, the oldest sessions stay open forever. In some cases these sessions might hold some resources inside WiredTiger, which can cause problems. eg: dhandles that never close in WiredTiger. This ticket is to build a mechanism around the session cache, to cleanup old sessions that have been idle for too long. More details in the linked tickets. |
| Comments |
| Comment by Sulabh Mahajan [ 05/Apr/19 ] |
|
bruce.lucas, WT session cache is NOT related to the WT cursor cache. I looked at 3.4 and we do have a WT session cache in there. The change here introduces a mechanism to close WT sessions when they have been idle for some time, mainly to encourage closing of active dhandles. |
| Comment by Bruce Lucas (Inactive) [ 04/Apr/19 ] |
|
sulabh.mahajan, can you clarify - is the wt session cache mentioned here related to the wt cursor caching mechanism that was introduced into 4.0 (and then backported to 3.6), or is this something different? In particular, could the session cache issue described here affect 3.4? |
| Comment by Sulabh Mahajan [ 03/Mar/19 ] |
|
Note: |
| Comment by Sulabh Mahajan [ 31/Jan/19 ] |
|
Update on backports: Backport to 4.0: Completed |
| Comment by Githook User [ 31/Jan/19 ] |
|
Author: {'name': 'Sulabh Mahajan', 'email': 'sulabh.mahajan@mongodb.com', 'username': 'sulabhM'}Message: (cherry picked from commit 97de2142f89ab280a4d0b2ddf168248c79f741d0) |
| Comment by Githook User [ 31/Jan/19 ] |
|
Author: {'name': 'Sulabh Mahajan', 'email': 'sulabh.mahajan@mongodb.com', 'username': 'sulabhM'}Message: (cherry picked from commit fb7a077a7a7d70aba2ca28564af0b0ccbee1f7fb) |
| Comment by Githook User [ 23/Jan/19 ] |
|
Author: {'email': 'sulabh.mahajan@mongodb.com', 'name': 'Sulabh Mahajan', 'username': 'sulabhM'}Message: (cherry picked from commit 97de2142f89ab280a4d0b2ddf168248c79f741d0) |
| Comment by Githook User [ 23/Jan/19 ] |
|
Author: {'username': 'sulabhM', 'email': 'sulabh.mahajan@mongodb.com', 'name': 'Sulabh Mahajan'}Message: (cherry picked from commit fb7a077a7a7d70aba2ca28564af0b0ccbee1f7fb) |
| Comment by Sulabh Mahajan [ 21/Jan/19 ] |
|
Updates on backports: Backport to 4.0: Approved - cherry picking and testing the backport now. |
| Comment by Jackie Chu [ 16/Jan/19 ] |
|
Excellent! Thank you! |
| Comment by Sulabh Mahajan [ 16/Jan/19 ] |
|
I have initiated the backport process to 4.0 and 3.6, the team will go over the change and if there are no concerns in backporting, we will get it done.
Killing the server sessions will not help here. This change handles cleanup of internal storage engine sessions that aren't exposed to the MongoDB user. |
| Comment by Githook User [ 16/Jan/19 ] |
|
Author: {'username': 'sulabhM', 'email': 'sulabh.mahajan@mongodb.com', 'name': 'Sulabh Mahajan'}Message: |
| Comment by Githook User [ 15/Jan/19 ] |
|
Author: {'username': 'sulabhM', 'email': 'sulabh.mahajan@mongodb.com', 'name': 'Sulabh Mahajan'}Message: |
| Comment by Jackie Chu [ 04/Jan/19 ] |
|
| Comment by Sulabh Mahajan [ 27/Dec/18 ] |
|
Proposal:
|