[SERVER-32706] Mongodb failing with message Got signal: 6 (Aborted) Created: 15/Jan/18 Updated: 02/Apr/18 Resolved: 09/Mar/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Admin |
| Affects Version/s: | 3.4.9 |
| Fix Version/s: | None |
| Type: | Question | Priority: | Major - P3 |
| Reporter: | Philip | Assignee: | Kelsey Schubert |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Participants: |
| Description |
|
My mongodb has been crashing repeatedly. I'm running my server on AWS using: I updated my ulimit to unlimited previously but I'm still receiving the same error. |
| Comments |
| Comment by Kelsey Schubert [ 09/Mar/18 ] | |||
|
Hi philip.leesha, We haven’t heard back from you for some time, so I’m going to mark this ticket as resolved. If this is still an issue for you, please provide additional information and we will reopen the ticket. Regards, | |||
| Comment by Kelsey Schubert [ 20/Feb/18 ] | |||
|
Hi philip.leesha, I just wanted to confirm that this issue has been resolved by upgrading your instance. Thanks, | |||
| Comment by Philip [ 26/Jan/18 ] | |||
|
Hi Mark, I checked the wired tiger cache size and it was set to use only about 40% of my total ram on my instance. However since this seems to be a persistent error I upgraded my instance so now I have a lot more ram to work with. I checked in with the wired tiger cache size on my upgraded instance and its still a percentage of my total ram. I'm hoping the increased memory leeway solves my issue. | |||
| Comment by Mark Agarunov [ 24/Jan/18 ] | |||
|
Hello philip.leesha, Thank you for the additional information. The out of memory error generally indicates that MongoDB is set to use more memory than is available on the server and was killed by the system once it had exhausted the memory, however if you provide the complete logs from the crash we can confirm that. To reduce memory usage, you can set the Wired Tiger cache size to a lower value. Thanks, | |||
| Comment by Philip [ 24/Jan/18 ] | |||
|
Hi Mark, My issue is still not resolved unfortunately. I've attached a new log failure. this time with a different kind of error: "out of memory."
| |||
| Comment by Mark Agarunov [ 19/Jan/18 ] | |||
|
Hello philip.leesha, Thank you for the additional information. I'll leave this ticket in "waiting for user input" for a few days and check in next week (unless you hit the error before then) so we can confirm that increasing the ulimits has fixed the problem. Thanks, | |||
| Comment by Philip [ 17/Jan/18 ] | |||
|
Hi Mark, I updated my ulimits to the recommended values. As mongodb was failing intermittently every few days, I'll have to wait a few days to see if the limit increase fixed my issue. Thank you for your help, | |||
| Comment by Mark Agarunov [ 16/Jan/18 ] | |||
|
Hello philip.leesha, Thank you for providing the additional info. My initial suggestion would be to attempt to increase the following ulimits:
to at least the recommended values. If this is still an issue after increasing the limits, please provide a description of how this node is set up:
Thanks, | |||
| Comment by Philip [ 16/Jan/18 ] | |||
|
Hi Mark, Thank you so much for your quick reply. I'm not sure if I increased the thread limits. Below are the ulimits for all limits set on my server. Here are the hard limits: (ulimit -H -a) | |||
| Comment by Mark Agarunov [ 16/Jan/18 ] | |||
|
Hello philip.leesha, Thank you for the report. Looking over the logs, there appears to be a significant amount of errors similar to pthread_create failed: Resource temporarily unavailable before the crash. Generally this indicates that the thread limit has been reached and the process cannot fork. When increasing ulimits, was the thread limit increased as well? If possible, please provide the ulimit parameters set for all limits. Thanks, |