[SERVER-34712] mongodb out of memory error Created: 27/Apr/18 Updated: 04/Jun/18 Resolved: 04/Jun/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Question | Priority: | Major - P3 |
| Reporter: | Guru chandra boopathi | Assignee: | Dmitry Agranat |
| Resolution: | Incomplete | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Participants: |
| Description |
|
MongoDB went down because out of memory. But I have enough ram space and disk space. I received the following error in log file. |
| Comments |
| Comment by Dmitry Agranat [ 04/Jun/18 ] | |||||||||||||||||||
|
Hi guruchandra, We haven’t heard back from you for some time, so I’m going to mark this ticket as resolved. If this is still an issue for you, please provide additional information and we will reopen the ticket. Regards, | |||||||||||||||||||
| Comment by Dmitry Agranat [ 27/May/18 ] | |||||||||||||||||||
|
Hi guruchandra, We still need the information I've requested in my last comment to diagnose the problem. If this is still an issue for you, can you please upload this data after enabling the heap profiler:
Thanks, | |||||||||||||||||||
| Comment by Dmitry Agranat [ 03/May/18 ] | |||||||||||||||||||
|
Hi guruchandra I've reviewed the data you've provided. mongod appears to be increasing its memory consumption outside of the WiredTiger cache. We've been able to rule out a number of common explanations for this behavior such as an increase in connections or a large number of open cursors, but do not have enough information to conclusively determine the root cause. To help us continue to investigate, would you be willing to restart an affected mongod with the heap profiler enabled and run it for a week or until the next occurrence of OOM?
This will allow us to identify which threads are accumulating memory. Please note that this is a diagnostic setting and we have observed a 10-30% performance impact on typical CPU-bound workloads. After reviewing the diagnostic.data, I believe that enabling this setting should not significantly harm your performance, and will provide the data to help us determine the root cause. However, to be safe, I would recommend more carefully monitoring the mongod after enabling the heap profiler. I've created a secure upload portal for you to use. After collecting the additional diagnostic.data please upload the following:
Lastly, could you clarify what other processes are running on this server? You can collect this information by executing top for a few minutes:
Thanks, | |||||||||||||||||||
| Comment by Guru chandra boopathi [ 02/May/18 ] | |||||||||||||||||||
|
Hi Dima, sorry , i was uploaded the wrong one.pls find the attachment now | |||||||||||||||||||
| Comment by Guru chandra boopathi [ 02/May/18 ] | |||||||||||||||||||
| Comment by Dmitry Agranat [ 01/May/18 ] | |||||||||||||||||||
|
Hi guruchandra The attached diagnostics.data is empty except for the awslogs-agent-setup.py file. Please archive the diagnostic.data directory under the node's $dbpath and upload it to this ticket. Thanks, | |||||||||||||||||||
| Comment by Guru chandra boopathi [ 30/Apr/18 ] | |||||||||||||||||||
|
hi Kelsey, Please find the attached backtrace and diagnostic.data . i don't have syslog for particular time. | |||||||||||||||||||
| Comment by Kelsey Schubert [ 27/Apr/18 ] | |||||||||||||||||||
|
Hi guruchandra, Thank you for reporting this issue. So we can continue to investigate, would you please provide the the following information?
Thanks again, | |||||||||||||||||||
| Comment by Guru chandra boopathi [ 27/Apr/18 ] | |||||||||||||||||||
|
Error message
|