Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-3755

mongos died unexpectedly

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 2.1.0
    • Affects Version/s: 1.8.3
    • Component/s: Stability
    • Labels:
      None
    • ALL

      Without any warnings or errors, two out of four mongos processes in our cluster suddenly died, all I can see in the mongos logs is "Received signal 6".

      Sat Sep  3 07:22:44 [conn5] ns: fragments_20110831.exposure_fragments Strategy::doQuery attempt: 0
      Sat Sep  3 07:22:44 [conn5] ns: fragments_20110831.exposure_fragments Strategy::doQuery attempt: 1
      Sat Sep  3 07:22:45 [conn5] ns: fragments_20110831.exposure_fragments Strategy::doQuery attempt: 2
      Sat Sep  3 07:22:47 [conn5] ns: fragments_20110831.exposure_fragments Strategy::doQuery attempt: 3
      Received signal 6
      Backtrace: 0x52f595 0x7f7e017e0af0 0x7f7e017e0a75 0x7f7e017e45c0 0x7f7e017d9941 0x69eb1c 0x5041eb 0x505aa4 0x6a7f70 0x
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo17printStackAndExitEi+0x75)[0x52f595]
      /lib/libc.so.6(+0x33af0)[0x7f7e017e0af0]
      /lib/libc.so.6(gsignal+0x35)[0x7f7e017e0a75]
      /lib/libc.so.6(abort+0x180)[0x7f7e017e45c0]
      /lib/libc.so.6(__assert_fail+0xf1)[0x7f7e017d9941]
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo17WriteBackListener3runEv+0x162c)[0x69eb1c]
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo13BackgroundJob7jobBodyEN5boost10shared_ptrINS0_9JobStatusEEE+0x12b)[0x5041eb]
      /opt/mongodb-1.8.3/bin/mongos(_ZN5boost6detail11thread_dataINS_3_bi6bind_tIvNS_4_mfi3mf1IvN5mongo13BackgroundJobENS_10
      /opt/mongodb-1.8.3/bin/mongos(thread_proxy+0x80)[0x6a7f70]
      /lib/libpthread.so.0(+0x69ca)[0x7f7e022e49ca]
      /lib/libc.so.6(clone+0x6d)[0x7f7e0189370d]
      ===
      Received signal 6
      Backtrace: 0x52f595 0x7f7e017e0af0 0x7f7e017e0a75 0x7f7e017e45c0 0x7f7e017d9941 0x530655 0x5c1ca3 0x581601 0x7f7e022e4
      
      Sat Sep  3 07:21:43 [conn6] creating ChunkManager ns: _id: "complete_20110902.exposures" took: 8ms sequenceNumber: 15
      Sat Sep  3 07:21:43 [conn6] creating ChunkManager ns: _id: "complete_20110902.pageviews" took: 2ms sequenceNumber: 16
      Received signal 6
      Backtrace: 0x52f595 0x7f851b1f0af0 0x7f851b1f0a75 0x7f851b1f45c0 0x7f851b1e9941 0x69eb1c 0x5041eb 0x505aa4 0x6a7f70 0x
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo17printStackAndExitEi+0x75)[0x52f595]
      /lib/libc.so.6(+0x33af0)[0x7f851b1f0af0]
      /lib/libc.so.6(gsignal+0x35)[0x7f851b1f0a75]
      /lib/libc.so.6(abort+0x180)[0x7f851b1f45c0]
      /lib/libc.so.6(__assert_fail+0xf1)[0x7f851b1e9941]
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo17WriteBackListener3runEv+0x162c)[0x69eb1c]
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo13BackgroundJob7jobBodyEN5boost10shared_ptrINS0_9JobStatusEEE+0x12b)[0x5041eb]
      /opt/mongodb-1.8.3/bin/mongos(_ZN5boost6detail11thread_dataINS_3_bi6bind_tIvNS_4_mfi3mf1IvN5mongo13BackgroundJobENS_10
      /opt/mongodb-1.8.3/bin/mongos(thread_proxy+0x80)[0x6a7f70]
      /lib/libpthread.so.0(+0x69ca)[0x7f851bcf49ca]
      /lib/libc.so.6(clone+0x6d)[0x7f851b2a370d]
      ===
      Received signal 11
      Backtrace: 0x52f595 0x7f851b1f0af0 0x532b40 0x57affc 0x5778f5 0x577da8 0x633240 0x63d6c9 0x63fec0 0x66b032 0x67fda7 0x
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo17printStackAndExitEi+0x75)[0x52f595]
      /lib/libc.so.6(+0x33af0)[0x7f851b1f0af0]
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo16DBConnectionPool11onHandedOutEPNS_12DBClientBaseE+0x20)[0x532b40]
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo17ClientConnections3getERKSsS2_b+0x1ac)[0x57affc]
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo15ShardConnection5_initEb+0x65)[0x5778f5]
      Received signal 11
      Backtrace: /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo15ShardConnectionC1ERKNS_5ShardERKSsb+0xa8)[0x577da8]
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo8Strategy6insertERKNS_5ShardEPKcRKNS_7BSONObjE+0x60)[0x633240]
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo13ShardStrategy7_insertERNS_7RequestERNS_9DbMessageEN5boost10shared_ptrINS_12Ch
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo13ShardStrategy7writeOpEiRNS_7RequestE+0x260)[0x63fec0]
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo7Request7processEi+0x172)[0x66b032]
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9L
      /opt/mongodb-1.8.3/bin/mongos(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x34d)[0x5815ed]
      /lib/libpthread.so.0(+0x69ca)[0x7f851bcf49ca]
      /lib/libc.so.6(clone+0x6d)[0x7f851b2a370d]
      ===
      0x52f595 0x7f851b1f0af0
      

            Assignee:
            greg_10gen Greg Studer
            Reporter:
            iconara Theo Hultberg
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: