Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-22831

Low query rate with heavy cache pressure and an idle collection

    XMLWordPrintable

    Details

    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Completed:
    • Case:

      Description

      • single-node replicat set, 3 GB cache, 20 GB oplog
      • insert 10 M x 1 kB documents (10 GB total size, plus index)
      • then 100 threads querying documents at random, observe low query rate
      • then restart mongod, same queries are now much faster

      • A-B: collection is being created
      • B-C: random queries
        • query rate is very low, ~6 k/s
        • rate of evicting from and reading into cache is ~9 k pages/s, ~1.5 pages per query, so very high miss ratio
        • rate of pages walked for eviction is very high, ~21 M/s, so about 2300 pages walked for every page evicted, or 2-3% of pages in cache walked for every page evicted
        • no pages are being evicted from oplog, but it is uncertain whether that is because all pages have already been evicted
      • C-D: after restart
        • query rate is much higher, ~34 k/s
        • rate of evicting from and reading into cache is 26 k pages/s, ~0.75 pages per query, so lower miss ratio than before restart
        • rate of pages walked for eviction is much lower

      The issue does not reproduce on a standalone node.

      Possibly related to SERVER-22423?

        Attachments

        1. diagnostic.data.tar
          95 kB
        2. queryrate.png
          queryrate.png
          243 kB
        3. two-collections.png
          two-collections.png
          117 kB

          Issue Links

            Activity

              People

              Assignee:
              michael.cahill Michael Cahill
              Reporter:
              bruce.lucas Bruce Lucas
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              22 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: