Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-7964

Server at 100% CPU, slow queries (15s) in spite of indexes

    • Type: Icon: Bug Bug
    • Resolution: Incomplete
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 2.0.1, 2.2.2
    • Component/s: None
    • None
    • Environment:
      Ubuntu 10.10 Server, Dual Core 2.7Ghz Xeon, 8GB RAM
    • ALL
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      Every 2 days, our primary instance hits 100% CPU usage and has to be restarted. By setting the profiling level to 1, I discovered that there are about a dozen EXTREMELY slow count() queries being completed per second:

      > db.system.profile.find()
      { "ts" : ISODate("2012-12-16T20:31:39.078Z"), "op" : "command", "ns" : "stylesaint.$cmd", "command" : { "count" : "tears", "query" : { "_id" :

      { "$gt" : ObjectId("50cdeadeaf58d3de96000294") }

      , "active" : true, "is_image_processed" : true, "hidden_from_feed" : false, "hidden_from_public_feeds" : false }, "fields" : null }, "ntoreturn" : 1, "responseLength" : 48, "millis" : 13930, "client" : "wouldntyouliketoknow" }
      ... other similar results, only the ObjectId is different

      I've run explain() on a number of these queries. Mongo seems to intelligently choose the index that gives it the fewest number of scanned documents, which is cool. But it means that one explain() does not explain them all. Here's an example:

      > db.tears.find({ "_id" :

      { "$gt" : ObjectId("50cdeadeaf58d3de96000294") }

      , "active" : true, "is_image_processed" : true, "hidden_from_feed" : false, "hidden_from_public_feeds" : false }).explain()
      {
      "cursor" : "BtreeCursor id",
      "isMultiKey" : false,
      "n" : 4,
      "nscannedObjects" : 5,
      "nscanned" : 5,
      "nscannedObjectsAllPlans" : 23,
      "nscannedAllPlans" : 25,
      "scanAndOrder" : false,
      "indexOnly" : false,
      "nYields" : 0,
      "nChunkSkips" : 0,
      "millis" : 0,
      "indexBounds" :

      { "_id" : [ [ ObjectId("50cdeadeaf58d3de96000294"), ObjectId("ffffffffffffffffffffffff") ] ] }

      ,
      "server" : "ip-10-169-65-66:27017"
      }

      I have not found any queries yet where nscanned is more than a few hundred documents.

      This particular collection has less than half a million documents. Whether or not these slow queries are related to the server failure, I'd like to speed them up. But my main issue is the server failure.

      I created a topic with the Google group. We can take the discussion there if you prefer: https://groups.google.com/forum/#!topic/mongodb-user/NbiWPSeYg28

            Assignee:
            Unassigned Unassigned
            Reporter:
            sebastian Sebastian Goodman
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: