Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16881

Pause due to sudden 4GB increase in cached bytes under WiredTiger

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 3.0.0-rc6
    • Affects Version/s: 2.8.0-rc5
    • Component/s: Storage
    • Labels:
    • Fully Compatible
    • ALL

      During one 10-minute run of a heavy mixed workload a period of several seconds of very low throughput was seen:

      • During the pause all statistics appear to be consistent with a large number of cache evictions, resulting in a lot of writes, system cpu time, and context switches. This heavy eviction load seems like a plausible cause for the low throughput during this period.
      • Immediately before the pause the number of bytes in the cache jumped by about 4 GiB between two samples about 0.1 seconds apart. This is a plausible trigger for the heavy eviction load.
      • During the pause bytes and pages in cache declined slowly, presumably a result of the page evictions.
      • The pause ends with a sudden drop in the size of the cache by about 4 GiB between two samples about 0.1 seconds apart.

      Here are the specific samples

      • just before increase
        "localTime":"2015-01-15T18:42:20.344Z"
        "bytes currently in the cache":4315898330
        "pages currently held in the cache":60756
      • just after increase
        "localTime":"2015-01-15T18:42:20.472Z"
        "bytes currently in the cache":8594367123
        "pages currently held in the cache":60598
      • just before decrease
        "localTime":"2015-01-15T18:42:25.058Z"
        "bytes currently in the cache":7242022268
        "pages currently held in the cache":35169
      • just after decrease
        "localTime":"2015-01-15T18:42:25.170Z"
        "bytes currently in the cache":2941126278
        "pages currently held in the cache":35137

      The size of the sudden jump was: 7242022268 - 2941126278 = 3.98 GiB

      The size of the sudden drop was: 8594367123 - 4315898330 = 4.01 GiB

      Neither a sudden increase nor a sudden decrease of cache size by 4 GiB in 0.1 seconds due to real cached data is plausible.

            Assignee:
            keith.bostic@mongodb.com Keith Bostic (Inactive)
            Reporter:
            bruce.lucas@mongodb.com Bruce Lucas (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: