Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-22831

Low query rate with heavy cache pressure and an idle collection

    XMLWordPrintable

Details

    • Fully Compatible
    • ALL

    Description

      • single-node replicat set, 3 GB cache, 20 GB oplog
      • insert 10 M x 1 kB documents (10 GB total size, plus index)
      • then 100 threads querying documents at random, observe low query rate
      • then restart mongod, same queries are now much faster

      • A-B: collection is being created
      • B-C: random queries
        • query rate is very low, ~6 k/s
        • rate of evicting from and reading into cache is ~9 k pages/s, ~1.5 pages per query, so very high miss ratio
        • rate of pages walked for eviction is very high, ~21 M/s, so about 2300 pages walked for every page evicted, or 2-3% of pages in cache walked for every page evicted
        • no pages are being evicted from oplog, but it is uncertain whether that is because all pages have already been evicted
      • C-D: after restart
        • query rate is much higher, ~34 k/s
        • rate of evicting from and reading into cache is 26 k pages/s, ~0.75 pages per query, so lower miss ratio than before restart
        • rate of pages walked for eviction is much lower

      The issue does not reproduce on a standalone node.

      Possibly related to SERVER-22423?

      Attachments

        1. diagnostic.data.tar
          95 kB
        2. queryrate.png
          queryrate.png
          243 kB
        3. two-collections.png
          two-collections.png
          117 kB

        Issue Links

          Activity

            People

              michael.cahill@mongodb.com Michael Cahill
              bruce.lucas@mongodb.com Bruce Lucas
              Votes:
              0 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: