[SERVER-6763] Random mongod Segmentation faults after getting Primary in Replica Set (mongo::BSONElement::toString) Created: 14/Aug/12 Updated: 15/Aug/12 Resolved: 14/Aug/12 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Replication, Stability |
| Affects Version/s: | 2.0.2, 2.0.7 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Nico Kaiser | Assignee: | Randolph Tan |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | bson, crash, replication | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Replica Set with 3 servers, each MongoDB 2.0.7 Linux 64-bit binaries, Debian GNU/Linux 6.0, Kernel 2.6.32-5-amd64, 4 GB RAM |
||
| Issue Links: |
|
||||||||
| Operating System: | Linux | ||||||||
| Participants: | |||||||||
| Description |
|
Suddenly our MongoDB setup (Replica Set with 3 servers) randomly crashes. It's always the PRIMARY server that segfaults after some time (3-5 hours), for no obvious reason. Then another server becomes PRIMARY and also crashes after some time. Stack trace:
The last log message before crashing is
but the PRIMARY server gets lots of these messages (and the setup ran well for some months now despite these messages). |
| Comments |
| Comment by Randolph Tan [ 14/Aug/12 ] | |||||||||||||||||||||||||||||||||||||||||||||
|
Not in 2.0.7, as it crashes when it tries to log that particular document. If it is possible for you to run version 2.2, then you would be able to see the document successfully being logged (truncated). You probably have to dig through the logs though, since the server will run smoothly. | |||||||||||||||||||||||||||||||||||||||||||||
| Comment by Nico Kaiser [ 14/Aug/12 ] | |||||||||||||||||||||||||||||||||||||||||||||
|
Ok I see. Is there a way to find the request / document that's causing the recursion? | |||||||||||||||||||||||||||||||||||||||||||||
| Comment by Nico Kaiser [ 14/Aug/12 ] | |||||||||||||||||||||||||||||||||||||||||||||
|
Ok, the primary server crashed again:
Latest log entries (logLevel 5):
| |||||||||||||||||||||||||||||||||||||||||||||
| Comment by Nico Kaiser [ 14/Aug/12 ] | |||||||||||||||||||||||||||||||||||||||||||||
|
I did not change anything. After the crashes began, I updated the server to 2.0.7 (from 2.0.2) and the (node.js) client. I'll turn up the logLevel and see if there is anything... | |||||||||||||||||||||||||||||||||||||||||||||
| Comment by Scott Hernandez (Inactive) [ 14/Aug/12 ] | |||||||||||||||||||||||||||||||||||||||||||||
|
Did you change anything in the config before this happened? Can you enable more verbose logging to catch more messages if it happens again? You can do this using the -v startup option or by using the setParameter command if the server is already running: http://docs.mongodb.org/manual/reference/commands/#setParameter |