Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-39658

Sudden CPU spike in secondary instances with no apparent cause

    XMLWordPrintable

    Details

    • Type: Question
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Performance
    • Labels:
      None

      Description

      Out of nowhere the two secondary instances for a three node cluster with very little CPU usage spiked to 100% and remained high.

      Version: 2.4.14

      Things we did so far:

      • Rebuilt all three databases from replicas.
      • reIndex()

      Things we discovered:

      Now that the load is higher and avg response time is >100ms than usual we see some query that should be using index not using it.

      Tue Feb 19 12:47:27.946 [conn11958] query locator.pages query: { query: { _depth:

      { $gte: 1, $lte: 1 }

      , _site: "sample.com" }, orderby: { _id: 1 } } ntoreturn:0 ntoskip:0 nscanned:2106803 keyUpdates:0 numYields: 11 locks(micros) r:5514254 nreturned:30 reslen:130866 2951ms

       

      Screenshots

       

      Secondary Node 1 - The other follows the same pattern.

      Primary Node (mostly idle except for nightly batch loads)

        Attachments

        1. image-2019-02-19-04-44-10-051.png
          image-2019-02-19-04-44-10-051.png
          100 kB
        2. image-2019-02-19-04-45-04-009.png
          image-2019-02-19-04-45-04-009.png
          69 kB
        3. image-2019-02-19-04-52-58-249.png
          image-2019-02-19-04-52-58-249.png
          59 kB
        4. mongodb-24-query-compare.JPG
          mongodb-24-query-compare.JPG
          115 kB

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: