Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-21782

WiredTiger freezing at 100% cache in use until service restart (daily)

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: None
    • Component/s: WiredTiger
    • Labels:
      None
    • Environment:
      Debian Wheezy.
      MongoDB 3.0.7
    • Linux

      On a daily basis we have to restart MongoDB nodes (secondaries and primary) because performance degrades dramatically.

      We can observe the percentage of WiredTiger cache (%use column when we use mongostat command) in use are freezing to 100% (and this percentage never decrease by it's own if we don't restart the node).

      After the node has been restarted, the %use decrease to ~80% generally.
      I've added an attachment of a graph displaying the %use on our servers.

      We tried to increase the cacheSize parameter.
      Here on a 64GB server:
      wiredTiger:
      engineConfig:
      cacheSizeGB: 56

      The issue is still the same even with this configuration.

        1. 21782.2632.jpg
          21782.2632.jpg
          100 kB
        2. 5jan16-logs.tar.gz
          1.89 MB
        3. capture-2016-01-09.tar.gz
          23.09 MB
        4. coll-stats-12jan16.tar.gz
          26 kB
        5. combined_usage-day.png
          combined_usage-day.png
          28 kB
        6. Dec18.png
          Dec18.png
          188 kB
        7. logging_20151208.tar.gz
          43.14 MB
        8. ls-lR-memc5-30dec15.txt.gz
          60 kB
        9. memc2-100pct-logs.txt
          64 kB
        10. mongodb_wt_cache_usage.png
          mongodb_wt_cache_usage.png
          48 kB
        11. mongostat-18dec15.log.gz
          39 kB
        12. sar.out
          3.57 MB
        13. server-status-18dec15.log.gz
          1.36 MB
        14. ss_mongodb.png
          ss_mongodb.png
          750 kB
        15. timeseries-2040.png
          timeseries-2040.png
          213 kB

            Assignee:
            ramon.fernandez@mongodb.com Ramon Fernandez Marina
            Reporter:
            anthony@stickyads.tv Anthony Pastor
            Votes:
            1 Vote for this issue
            Watchers:
            15 Start watching this issue

              Created:
              Updated:
              Resolved: