High CPU (user usage in MMS) after bulk inserts

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Done
    • Priority: Major - P3
    • None
    • Affects Version/s: 3.0.4
    • Component/s: Admin
    • None
    • ALL
    • Hide

      Start a fresh replica set,
      Start few bulk inserts,
      The CPU usage goes over 100% and remains there until next restart.

      Show
      Start a fresh replica set, Start few bulk inserts, The CPU usage goes over 100% and remains there until next restart.
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Hello,

      I have a replica set MongoDB 3.0.4 with stable load (60-100 inserts, 20-30 updates, 10 queries). Sometimes I do bulk inserts like 20K 30K and after this the CPU goes high even though I am done with the operation and there is nothing else.

      In MMS when I look at the CPU of a fresh MongoDB (just started) everything is normal. But when I use bulk write to insert 20k or 40k or any job that uses cache the CPU usage (specially user usage) goes over 100%. The servers are isolated from any other applications and they only use MongoDB and I check with other monitoring tools it's all MongoDB. When I restart the MongoDB it all goes away.

      How I start my replica:

      mongod --storageEngine wiredTiger --wiredTigerCacheSizeGB=2 --dbpath /data/ --replSet rs0 --fork --logpath /data/fork.log

      I also tried different cache size like 5 or 10 and it was the same. The only thing that looks different before bulk inserts and after bulk inserts is the cache usage in MMS.

      Stats:

      rs0:PRIMARY> db.stats()
      {
              "db" : "test",
              "collections" : 66,
              "objects" : 1943772773,
              "avgObjSize" : 741.1523292167237,
              "dataSize" : 1440631718177,
              "storageSize" : 736019369984,
              "numExtents" : 0,
              "indexes" : 193,
              "indexSize" : 27145035776,
              "ok" : 1
      }
      
                             insert query update delete getmore command % dirty % used flushes vsize  res qr|qw ar|aw netIn netOut conn set repl     time
             localhost:27017    *33     3    *10     *0       0     9|0     6.1   33.8       0  5.8G 5.2G   0|3   1|0    1k   407k  112 rs0  SEC 10:38:14
      mongodb-replica1:27017     31    *0     18     *0      34     8|0    22.0   50.5       0  2.2G 1.5G   0|1   1|0   50k    62k  107 rs0  PRI 10:38:14
      mongodb-replica2:27017    *33     3    *10     *0       0    10|0     6.1   33.8       0  5.8G 5.2G   0|3   1|0    1k   407k  112 rs0  SEC 10:38:14
      
             localhost:27017    *41    10    *14     *0       0    10|0     6.1   33.8       0  5.8G 5.2G   0|3   1|0    3k   386k  112 rs0  SEC 10:38:15
      mongodb-replica1:27017     39    *0     14     *0      42    11|0    22.0   50.5       0  2.2G 1.5G   0|1   1|0   54k    68k  107 rs0  PRI 10:38:15
      mongodb-replica2:27017    *41    10    *14     *0       0     9|0     6.1   33.8       0  5.8G 5.2G   0|3   2|0    3k   386k  112 rs0  SEC 10:38:15
      
             localhost:27017    *45     6    *19     *0       0    14|0     6.1   33.8       0  5.8G 5.2G   0|3   2|0    2k   223k  112 rs0  SEC 10:38:16
      mongodb-replica1:27017     34    *0     19     *0      36    18|0    22.0   50.5       0  2.2G 1.5G   0|0   1|1   50k    64k  107 rs0  PRI 10:38:16
      mongodb-replica2:27017    *45     6    *19     *0       0    14|0     6.1   33.8       0  5.8G 5.2G   0|3   2|0    2k   223k  112 rs0  SEC 10:38:16
      

      Please let me know if you need more information.

      Thanks

        1. Screen Shot 2015-06-30 at 13.44.16.png
          Screen Shot 2015-06-30 at 13.44.16.png
          162 kB
        2. screenshot-1.png
          screenshot-1.png
          315 kB
        3. screenshot-2.png
          screenshot-2.png
          306 kB
        4. screenshot-3.png
          screenshot-3.png
          332 kB
        5. screenshot-4.png
          screenshot-4.png
          304 kB
        6. screenshot-5.png
          screenshot-5.png
          321 kB
        7. screenshot-6.png
          screenshot-6.png
          337 kB
        8. screenshot-7.png
          screenshot-7.png
          338 kB
        9. Screen Shot 2015-07-01 at 10.33.46.png
          Screen Shot 2015-07-01 at 10.33.46.png
          180 kB
        10. Screen Shot 2015-08-31 at 22.28.04.png
          Screen Shot 2015-08-31 at 22.28.04.png
          270 kB

            Assignee:
            Sam Kleinman (Inactive)
            Reporter:
            Maziyar Panahi
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: