[SERVER-22752] WiredTiger reach max 20010 sessions: Cannot allocate memory Created: 19/Feb/16 Updated: 30/Mar/16 Resolved: 19/Feb/16 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | WiredTiger |
| Affects Version/s: | 3.0.8, 3.0.9 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Critical - P2 |
| Reporter: | Dror Bar-Gil | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Operating System: | ALL |
| Participants: |
| Description |
|
Hi All, The reason: It indicates we were hitting the 20k session limit, we understand that it might be due to not properly closing cursors but it seems to happen too often and cause the primary to shutdown instead of reject the new sessions. I start all the servers again and this is the number I had after the 4th mongo was up,
When I tried to load the 5th one it failed for exceeding the limit of 20010 sessions It was obvious that there were too many open sessions, in order to close them manually. I decided to set the timeout to 5Sec by running
It seems that the default was 600 sec! After doing it and 5 sec later I run the command again and found that number of sessions decreased to 357
I upload the last mongo and everything looks fine now. It is same behavior as described by Glen Miner at I attached the all information of our server (output of mdiag.sh) and snippet of the log from time of the crash. meanwhile I set the default cursor time our to 30 sec just to be on the safe side even though 5 sec seems OK as well. |
| Comments |
| Comment by Ramon Fernandez Marina [ 06/Mar/16 ] | |||||
|
Here's how to add this parameter to a config file:
As for your second comment, at the moment MongoDB does not have the admission control features you describe. Implementing these features is a difficult challenge, bug feel free to open a new feature request ticket. Note also that the changes in Regards, | |||||
| Comment by Dror Bar-Gil [ 19/Feb/16 ] | |||||
|
Hi Ramon, Two comments: 2. As a result of this case we intend to go over all our code and make sure any cursor is closed. | |||||
| Comment by Ramon Fernandez Marina [ 19/Feb/16 ] | |||||
|
dror@amobee.com, there's a default limit of 20000 sessions in WiredTiger. You may want to review your application to make sure unused cursors are closed to avoid consuming memory ( One alternative, as you've found out, is to lower the default cursor timeout (documented here). Another option is to increase the maximum number of sessions:
Note however that each open session consumes memory, so it's good practice to close unused cursors. Regards, |