Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17862

Unify stats reporting for log/profiler across CRUD operations

XMLWordPrintableJSON

    • Type: Icon: Improvement Improvement
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 3.3.9
    • Affects Version/s: None
    • Component/s: Diagnostics, Logging
    • None
    • Major Change
    • Query 15 (06/03/16), Query 16 (06/24/16)

      Issue Status as of Jun 08, 2016

      ISSUE SUMMARY
      This project improves the existing MongoDB diagnostics facilities by ensuring that the same diagnostic information is available across all operations.

      Consistent reporting of metrics across user operations

      All relevant metrics are supported for each command and operation (with a small list of exceptions below). These metrics are available in the slow diagnostic log, the profiler and db.currentOp().

      Operations included

      1. count
      2. distinct
      3. find (command + OP_QUERY)
      4. findAndModify
      5. getMore (command + OP_GET_MORE)
      6. geoNear
      7. group
      8. mapReduce
      9. delete (command + OP_DELETE)
      10. update (command + OP_UPDATE)

      Metrics covered

      1. keysExamined
      2. docsExamined
      3. reslen
      4. nreturned
      5. ntoreturn
      6. batchSize
      7. numYields
      8. cursorExhausted
      9. planSummary
      10. cursorid
      11. writeConflicts
      12. hasSortStage
      13. fromMultiPlanner
      14. replanned
      15. locks
      16. protocol
      17. millis
      18. ninserted (or 'upsert' for updates that result in an insert)
      19. multi
      20. nMatched
      21. nModified
      22. ndeleted
      23. ntoskip
      24. exhaust
      25. nmoved
      26. upsert
      27. execStats

      Exceptions

      1. execStats is only available in the profiler output. It is not available in the slow diagnostics log or db.currentOp().
      2. execStats is not available for the aggregate command.
      Introduction of new metrics

      This project introduces 2 new metrics, 'keysInserted' and 'keysDeleted'. These metrics report the number of index keys inserted and deleted for all write operations. This replaces 'keyUpdates' which was not previously covered across write commands and is potentially misleading.

      Removed metrics

      This project removes the following metrics:

      1. fastmod: replaced by 'keysInserted', 'keysDeleted' and 'nmoved', which in combination can be used to derive this metric in a more granular, easy to understand manner.
      2. keyUpdates: replaced by 'keysInserted' and 'keysDeleted'.
      3. idhack: replaced by 'planSummary', which will report when the IDHACK stage is used.
      4. moved: obsoleted by the 'nmoved' metric.

      See SERVER-23272 for the rationale behind this removal.

      Deprecated metrics
      1. fastmodinsert: not considered generally useful for performance analysis.
      Original description
      Goals
      • Improve existing diagnostics by ensuring that the same information is available across operations (e.g. we don't report planSummary for all operations).
      • In other words, expose existing debug information more consistently.
      Non-goals
      • Collect additional debug information which we don't already collect.
      • Major infrastructural changes to CurOp, the profiler, or the slow query log.
      How

      Part of the work for SERVER-10448 (a.k.a. "new explain"), completed for version 3.0, was to add a suite of functions capable of pulling summary statistics and debug information out of PlanExecutor (see explain.h). However, we use these functions in only a few places instead of uniformly populating CurOp with the available debug information. Using explain functionality to add the same stats to CurOp everywhere should be an easy win for diagnosability.

      Details

      This includes work for the following operations:

      • count
      • delete
      • distinct
      • find (OP_QUERY and command)
      • findAndModify
      • geoNear
      • getMore (OP_GET_MORE and command)
      • group
      • update

      The following fields will be added to CurOp for every operation once the operation completes:

      • docsExamined
      • hasBlockingSort
      • idhack
      • keysExamined
      • nReturned
      • planSummary

            Assignee:
            james.wahlin@mongodb.com James Wahlin
            Reporter:
            ian@mongodb.com Ian Whalen (Inactive)
            Votes:
            2 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: