[SERVER-18336] mongod crash Created: 06/May/15  Updated: 13/May/15  Resolved: 11/May/15

Status: Closed
Project: Core Server
Component/s: Stability
Affects Version/s: 2.4.10
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: DaixiShi Assignee: Sam Kleinman (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: JPEG File QQ图片20150508094505.jpg     PNG File QQ截图20150506161037.png    
Issue Links:
Duplicate
Participants:

 Description   

Our environment is centOS 7 with hyper-v, 64bit.
Our mongod process often crashed with no logs recorded



 Comments   
Comment by Sam Kleinman (Inactive) [ 11/May/15 ]

Thanks for this information. When processes interact with the OOM Killer, the only courses of action are to either increase the available memory, or attempting to restrict the amount of memory that the killed application uses. MongoDB does not support TokuMX. Please contact your TokuMX vendor for help with TokuMX.

Regards,
sam

Comment by DaixiShi [ 08/May/15 ]

Thanks for Kleinman's help!

Our release is tokumx 2.0.0 based on MongoDB release 2.4.10.

We deployed a sharded cluster and each cluster is a replication set.

The crashed instance was always the primary node and no logs were recorded.

We checked the system log and found that the crashed mongod process requested more memory. The memory had almost reached the limit and the system oom_killer killed this mongod process.

Comment by DaixiShi [ 08/May/15 ]

oom_killer

Comment by Sam Kleinman (Inactive) [ 07/May/15 ]

Thanks. First, I just want to confirm that you've encountered this issue on MongoDB 2.4.10.

  • Can you provide more information about the instance that crashed. Is it a member of a replica set and/or sharded cluster? If so what was its role in this deployment (primary, secondary, config server, etc.)
  • Have you attempted to upgrade to the latest release of 2.4? There may be additional stability features in these release. You may also want to consider upgrading to MongoDB 2.6 or 3.0, which may include fixes that could resolve this issue.
  • How often do you observe the crash in your environment?
  • Do you have any additional information about the operations that correlate with the crash? What was the workload of the instance generally, and during the time of the crash?
  • Any logging that you've captured will help us understand the cause of the crash. Please attach the mongod.log file to the ticket which will contain a copy of the stacktrace which will
    help us pinpoint the cause of the crash.

Our next steps here are to understand the cause of the crash, and attempt to reproduce the crash in a controlled environment and then to attempt to reproduce the crash on more recent versions of MongoDB. With luck, this issue will have already been resolved by earlier changes; but if not having a reproduction will allow us to fix the issue.

I look forward to hearing from you and getting to look at the logs.

Regards,
sam

Generated at Thu Feb 08 03:47:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.