Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-15580

Evaluating candidate query plans with concurrent writes on same collection may crash mongod

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical - P2
    • Resolution: Fixed
    • 2.6.3
    • 2.6.6, 2.8.0-rc0
    • Querying
    • None
    • ALL

    Description

      Our MongoDB server (3 servers, in replica set) was running normally until suddenly the primary segfaulted. This is what's in the log (obviously, just the tail of the log):

      2014-10-09T14:38:18.441+0300 [conn11661296] killcursors  keyUpdates:0 numYields:0 locks(micros) r:212 127ms
      2014-10-09T14:38:18.450+0300 [conn11667693] query crossbow_cos.node query: { $query: { _bundle: "xb4_m4_data_video", _type: "node", cos_deleted: 0, status: 1, _id: { $in: [ 1313690, 1250602, 1314508, 1300482, 933372, 1304961, 1311114, 533803, 1315776 ] }, cos_field_publishing_channels: "nelonenmedia", cos_private_tags: null }, orderby: { cos_created: -1 } } planSummary: IXSCAN { _id: 1 }, IXSCAN { _id: 1 } ntoreturn:10 ntoskip:0 nscanned:34 nscannedObjects:18 keyUpdates:0 numYields:2 locks(micros) r:18881 nreturned:7 reslen:52504 1025ms
      2014-10-09T14:38:18.513+0300 [conn11661120] killcursors: found 0 of 1
      2014-10-09T14:38:18.593+0300 [conn11661509] SEVERE: Invalid access at address: 0
      2014-10-09T14:38:18.694+0300 [conn11667971] killcursors: found 0 of 1
      2014-10-09T14:38:18.990+0300 [conn11661509] SEVERE: Got signal: 11 (Segmentation fault).
      Backtrace:0x11da801 0x11d9bde 0x11d9ccf 0x3b8000f710 0xaa7241 0xd4cccc 0xd4d69d 0xd4f81e 0xc12931 0xa37c44 0xa2295a 0xa241b7 0xa26269 0xd549f7 0xb9c972 0xb9ef52 0x76f2bf 0x1190afb 0x3b800079d1 0x3b7fce8b6d 
       /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0x11da801]
       /usr/bin/mongod() [0x11d9bde]
       /usr/bin/mongod() [0x11d9ccf]
       /lib64/libpthread.so.0() [0x3b8000f710]
       /usr/bin/mongod(_ZN5mongo18KeepMutationsStage4workEPm+0xc1) [0xaa7241]
       /usr/bin/mongod(_ZN5mongo15MultiPlanRunner12workAllPlansEPNS_7BSONObjEm+0x13c) [0xd4cccc]
       /usr/bin/mongod(_ZN5mongo15MultiPlanRunner12pickBestPlanEPmPNS_7BSONObjE+0xed) [0xd4d69d]
       /usr/bin/mongod(_ZN5mongo15MultiPlanRunner7getNextEPNS_7BSONObjEPNS_7DiskLocE+0x34e) [0xd4f81e]
       /usr/bin/mongod(_ZN5mongo8runCountERKSsRKNS_7BSONObjERSsRi+0xe11) [0xc12931]
       /usr/bin/mongod(_ZN5mongo8CmdCount3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x114) [0xa37c44]
       /usr/bin/mongod(_ZN5mongo12_execCommandEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x3a) [0xa2295a]
       /usr/bin/mongod(_ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xe87) [0xa241b7]
       /usr/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x6c9) [0xa26269]
       /usr/bin/mongod(_ZN5mongo11newRunQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x2307) [0xd549f7]
       /usr/bin/mongod() [0xb9c972]
       /usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x442) [0xb9ef52]
       /usr/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x9f) [0x76f2bf]
       /usr/bin/mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x4fb) [0x1190afb]
       /lib64/libpthread.so.0() [0x3b800079d1]
       /lib64/libc.so.6(clone+0x6d) [0x3b7fce8b6d]

      If we caused this somehow, any advice how to avoid it would be hugely useful

      Attachments

        Issue Links

          Activity

            People

              rassi J Rassi
              kvirta Kalle Varisvirta
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: