[SERVER-16213] Mongod crashes after running mongodump Created: 18/Nov/14  Updated: 18/Nov/14  Resolved: 18/Nov/14

Status: Closed
Project: Core Server
Component/s: Stability
Affects Version/s: 2.6.5
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Martin de Vries Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Debian Linux 7, using the http://downloads-distro.mongodb.org/repo/debian-sysvinit repository


Operating System: Linux
Steps To Reproduce:

I think our collection is needed to reproduce the problem

Participants:

 Description   

Our mongod (single server instance) crashes often (but not always) after running a nightly backup with mongodump. The mongodump command executes succesfully, but after that new connections fail (connection timeout). The only thing we can do is to restart the server.

Some information:

  • There is no information in de log files (the last entry is the connection of the backup user)
  • The application mainly uses one collection with about 500.000 documents
  • The problem occurs since there is traffic on the application, there wasn't a problem before that. During the run of mongodump there are concurrent reads from the database
  • We did try a database repair a couple of times, but this didn't help
  • The only this we changed in de default configuration is that we turned authentication on


 Comments   
Comment by Martin de Vries [ 18/Nov/14 ]

Thank you very much for your reply, but we just found out the problem ourselves.

We already have this problem for a several months and did much investigation, but we now found out that it's our own watchdog that kill mongo because it uses more memory than allowed.

Sorry for disturbing you. The issue can be closed.

Comment by Asya Kamsky [ 18/Nov/14 ]

Also, you mention frequent crashes - do you mean mongod crashes (and it's down until you restart it)?

Or do you mean it becomes non-responsive but there are no crashes or stack traces in the logs?

Comment by Asya Kamsky [ 18/Nov/14 ]

Have you tried increasing the log level and seeing what's in the logs after the backup starts?

db.adminCommand({setParameter:1,logLevel:1})

Can you provide approximate stats - number of databases, sizes of them? How much physical RAM is on the machine? Is it possible the mongodump run exhausts available physical resources and the subsequent requests are simply very slow?

What driver and version is the application using to connect to mongod?

Generated at Thu Feb 08 03:40:19 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.