Without any warnings or errors, two out of four mongos processes in our cluster suddenly died, all I can see in the mongos logs is "Received signal 6".
Sat Sep 3 07:22:44 [conn5] ns: fragments_20110831.exposure_fragments Strategy::doQuery attempt: 0 Sat Sep 3 07:22:44 [conn5] ns: fragments_20110831.exposure_fragments Strategy::doQuery attempt: 1 Sat Sep 3 07:22:45 [conn5] ns: fragments_20110831.exposure_fragments Strategy::doQuery attempt: 2 Sat Sep 3 07:22:47 [conn5] ns: fragments_20110831.exposure_fragments Strategy::doQuery attempt: 3 Received signal 6 Backtrace: 0x52f595 0x7f7e017e0af0 0x7f7e017e0a75 0x7f7e017e45c0 0x7f7e017d9941 0x69eb1c 0x5041eb 0x505aa4 0x6a7f70 0x /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo17printStackAndExitEi+0x75)[0x52f595] /lib/libc.so.6(+0x33af0)[0x7f7e017e0af0] /lib/libc.so.6(gsignal+0x35)[0x7f7e017e0a75] /lib/libc.so.6(abort+0x180)[0x7f7e017e45c0] /lib/libc.so.6(__assert_fail+0xf1)[0x7f7e017d9941] /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo17WriteBackListener3runEv+0x162c)[0x69eb1c] /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo13BackgroundJob7jobBodyEN5boost10shared_ptrINS0_9JobStatusEEE+0x12b)[0x5041eb] /opt/mongodb-1.8.3/bin/mongos(_ZN5boost6detail11thread_dataINS_3_bi6bind_tIvNS_4_mfi3mf1IvN5mongo13BackgroundJobENS_10 /opt/mongodb-1.8.3/bin/mongos(thread_proxy+0x80)[0x6a7f70] /lib/libpthread.so.0(+0x69ca)[0x7f7e022e49ca] /lib/libc.so.6(clone+0x6d)[0x7f7e0189370d] === Received signal 6 Backtrace: 0x52f595 0x7f7e017e0af0 0x7f7e017e0a75 0x7f7e017e45c0 0x7f7e017d9941 0x530655 0x5c1ca3 0x581601 0x7f7e022e4
Sat Sep 3 07:21:43 [conn6] creating ChunkManager ns: _id: "complete_20110902.exposures" took: 8ms sequenceNumber: 15 Sat Sep 3 07:21:43 [conn6] creating ChunkManager ns: _id: "complete_20110902.pageviews" took: 2ms sequenceNumber: 16 Received signal 6 Backtrace: 0x52f595 0x7f851b1f0af0 0x7f851b1f0a75 0x7f851b1f45c0 0x7f851b1e9941 0x69eb1c 0x5041eb 0x505aa4 0x6a7f70 0x /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo17printStackAndExitEi+0x75)[0x52f595] /lib/libc.so.6(+0x33af0)[0x7f851b1f0af0] /lib/libc.so.6(gsignal+0x35)[0x7f851b1f0a75] /lib/libc.so.6(abort+0x180)[0x7f851b1f45c0] /lib/libc.so.6(__assert_fail+0xf1)[0x7f851b1e9941] /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo17WriteBackListener3runEv+0x162c)[0x69eb1c] /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo13BackgroundJob7jobBodyEN5boost10shared_ptrINS0_9JobStatusEEE+0x12b)[0x5041eb] /opt/mongodb-1.8.3/bin/mongos(_ZN5boost6detail11thread_dataINS_3_bi6bind_tIvNS_4_mfi3mf1IvN5mongo13BackgroundJobENS_10 /opt/mongodb-1.8.3/bin/mongos(thread_proxy+0x80)[0x6a7f70] /lib/libpthread.so.0(+0x69ca)[0x7f851bcf49ca] /lib/libc.so.6(clone+0x6d)[0x7f851b2a370d] === Received signal 11 Backtrace: 0x52f595 0x7f851b1f0af0 0x532b40 0x57affc 0x5778f5 0x577da8 0x633240 0x63d6c9 0x63fec0 0x66b032 0x67fda7 0x /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo17printStackAndExitEi+0x75)[0x52f595] /lib/libc.so.6(+0x33af0)[0x7f851b1f0af0] /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo16DBConnectionPool11onHandedOutEPNS_12DBClientBaseE+0x20)[0x532b40] /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo17ClientConnections3getERKSsS2_b+0x1ac)[0x57affc] /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo15ShardConnection5_initEb+0x65)[0x5778f5] Received signal 11 Backtrace: /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo15ShardConnectionC1ERKNS_5ShardERKSsb+0xa8)[0x577da8] /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo8Strategy6insertERKNS_5ShardEPKcRKNS_7BSONObjE+0x60)[0x633240] /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo13ShardStrategy7_insertERNS_7RequestERNS_9DbMessageEN5boost10shared_ptrINS_12Ch /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo13ShardStrategy7writeOpEiRNS_7RequestE+0x260)[0x63fec0] /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo7Request7processEi+0x172)[0x66b032] /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9L /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x34d)[0x5815ed] /lib/libpthread.so.0(+0x69ca)[0x7f851bcf49ca] /lib/libc.so.6(clone+0x6d)[0x7f851b2a370d] === 0x52f595 0x7f851b1f0af0
- is duplicated by
-
SERVER-3754 Seeing lots of setShardVersion failed
- Closed
-
SERVER-4850 SetShardVersion failed: client version differs from config's for collection "xxx" with multiple mongos
- Closed
- related to
-
SERVER-3753 "db config reload failed"
- Closed