Details
-
Bug
-
Status: Closed
-
Major - P3
-
Resolution: Incomplete
-
2.2.2
-
None
-
None
-
EC2 XL Linux instance
-
Linux
Description
We've experienced several crashes of our production mongod. The server is under reasonable load, with queries backed by indices and no long running queries. Here's one of the crashes:
Sat Feb 9 14:47:39 Backtrace:
0xaffd31 0x558bb9 0x559142 0x7fbf0f8f6500 0x7fbf0ecea60e 0x5c3be8 0x8f480b 0x901a0b 0x908769 0x909072 0x9097c7 0x8dda17 0x8de446 0x8e751a 0x8e8a78 0x8e8c3b 0x82bde4 0x6f1072 0x6e8d71 0x6ea700
/usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xaffd31]
/usr/bin/mongod(_ZN5mongo10abruptQuitEi+0x399) [0x558bb9]
/usr/bin/mongod(_ZN5mongo24abruptQuitWithAddrSignalEiP7siginfoPv+0x262) [0x559142]
/lib64/libpthread.so.0(+0xf500) [0x7fbf0f8f6500]
/lib64/libc.so.6(+0x13d60e) [0x7fbf0ecea60e]
/usr/bin/mongod(ZN5mongo20compareElementValuesERKNS_11BSONElementES2+0x778) [0x5c3be8]
/usr/bin/mongod(_ZN5mongo10FieldRange9intersectERKS0_b+0x12b) [0x8f480b]
/usr/bin/mongod(_ZN5mongo13FieldRangeSet19intersectMatchFieldEPKcRKNS_11BSONElementEbb+0x5b) [0x901a0b]
/usr/bin/mongod(_ZN5mongo13FieldRangeSet16handleMatchFieldERKNS_11BSONElementEb+0x1b9) [0x908769]
/usr/bin/mongod(_ZN5mongo13FieldRangeSet4initEb+0xa2) [0x909072]
/usr/bin/mongod(_ZN5mongo13FieldRangeSetC1EPKcRKNS_7BSONObjEbb+0x147) [0x9097c7]
/usr/bin/mongod(ZN5mongo16MultiPlanScanner4initERKNS_7BSONObjES3_S3+0x87) [0x8dda17]
/usr/bin/mongod(ZN5mongo16MultiPlanScanner4makeEPKcRKNS_7BSONObjES5_RKN5boost10shared_ptrIKNS_11ParsedQueryEEES5_NS_18QueryPlanGenerator18RecordedPlanPolicyES5_S5+0x76) [0x8de446]
/usr/bin/mongod(_ZN5mongo15CursorGenerator19setMultiPlanScannerEv+0xea) [0x8e751a]
/usr/bin/mongod(_ZN5mongo15CursorGenerator8generateEv+0x98) [0x8e8a78]
/usr/bin/mongod(_ZN5mongo25NamespaceDetailsTransient9getCursorEPKcRKNS_7BSONObjES5_RKNS_24QueryPlanSelectionPolicyEPbRKN5boost10shared_ptrIKNS_11ParsedQueryEEEbPNS_16QueryPlanSummaryE+0x4b) [0x8e8c3b]
/usr/bin/mongod(_ZN5mongo8runCountEPKcRKNS_7BSONObjERSsRi+0x494) [0x82bde4]
/usr/bin/mongod(_ZN5mongo8CmdCount3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x62) [0x6f1072]
/usr/bin/mongod(_ZN5mongo12_execCommandEPNS_7CommandERKSsRNS_7BSONObjEiRNS_14BSONObjBuilderEb+0x51) [0x6e8d71]
/usr/bin/mongod(_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xe70) [0x6ea700]
In case it's relevant, we have 3 replication instances: a primary, a secondary, and an arbiter, all within the same ec2 zone.