[SERVER-31344] Process limit not handled gracefully Created: 30/Sep/17 Updated: 27/Oct/23 Resolved: 17/Oct/17 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Networking |
| Affects Version/s: | 3.4.3 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Siarhiej Jaskievi? | Assignee: | Andrew Morrow (Inactive) |
| Resolution: | Works as Designed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||||||||
| Steps To Reproduce: | Set ulimit -u to 1k. Added some requests to mongodb, opening new connections (near 1k)
This was happening with every replica in replicaset (because of constant number of requests), replicaset became unstable. |
||||||||||||||||||||||||||||||||
| Participants: |
| Description |
|
Hi! |
| Comments |
| Comment by Andrew Morrow (Inactive) [ 17/Oct/17 ] |
|
Hi yasek - I'm closing this ticket since it doesn't seem to have any additional activity. Please feel free to re-open or add additional comments if there is any additional help I can provide. |
| Comment by Andrew Morrow (Inactive) [ 09/Oct/17 ] |
|
yasek - I'd recommend over-provisioning w.r.t. to any ulimits. We do not currently attempt to gracefully degrade when we are close to violating them. We treat ulimits as quotas that the user has chosen to enforce on us, and consider a violation of those limits as an indication that the system is misbehaving and should be terminated by the OS as the next level of error handling. Attempting to set the ulimits at or just above your expected capacity means that transient events could push you into failure. I would advise dropping ulimit restrictions entirely or setting them to a constant multiple of your expected maximum. To answer your specific question, you are not safe to handle 64k connections with a ulimit set at 64k processes, as mongodb can and does make use of background threads. Please let me know if there is any additional information I can provide to help you set appropriate values. |
| Comment by Kelsey Schubert [ 06/Oct/17 ] |
|
Hi yasek, If you expect that your that your workload will generate close to 64k connections, I would recommend increasing your limits to ensure that mongod can continue to access required resources under load. I've modified the ticket to summary to describe work to enable mongod to more gracefully approach these limits and marked it to be considered by our Platforms Team - please continue to watch for updates. Kind regards, |
| Comment by Siarhiej Jaskievi? [ 03/Oct/17 ] |
|
anonymous.user, any suggestions? |
| Comment by Siarhiej Jaskievi? [ 30/Sep/17 ] |
|
Yes, I realize, that increasing ulimits could resolve this issue on such workload. |
| Comment by Kelsey Schubert [ 30/Sep/17 ] |
|
Hi yasek, Thank you for the report. mongod requires ulimits to be appropriately set for its workload. Please increase your settings to our recommended values and reevaluate the performance of your replica set to confirm that the issue has been resolved. Kind regards, |