Investigate changes in SERVER-109983: Reorganize $queryStats output with more structure

    • Type: Investigation
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Tools and Replicator
    • 0

      Original Downstream Change Summary

      The output of $queryStats is expected to change. See Description of SERVER-109983 to see how the new output format is expected to change. It will be controlled by a feature flag, QueryStatsMetricsSubsections, which is currently off.

      Description of Linked Ticket

      As part of adding write metrics like 'nUpdated', we are wondering about changing the output schema of $queryStats. Going from something like this (current structure):

      {
          key: {      /* same */     },
          keyHash: 'dsoJ+LHAru0z6MJ1/IygJnnLTrlpVYYmPnlmNZbZrLI=',
          queryShapeHash: "uxMLCvpiJ5N/IRqt4c28/0A8F01C8AA16CA805FF5C1A5737535F97E40C2A90CE91A82CCB7A74C7CCB9C48",
          "metrics" : {
                      "lastExecutionMicros" : NumberLong(903),
                      "execCount" : NumberLong(1),
                      "totalExecMicros" : {
                              "sum" : NumberLong(903),
                              "max" : NumberLong(903),
                              "min" : NumberLong(903),
                              "sumOfSquares" : NumberDecimal("815409")
                      },
                      "firstResponseExecMicros" : {
                              "sum" : NumberLong(903),
                              "max" : NumberLong(903),
                              "min" : NumberLong(903),
                              "sumOfSquares" : NumberDecimal("815409")
                      },
                      "docsReturned" : {
                              "sum" : NumberLong(1),
                              "max" : NumberLong(1),
                              "min" : NumberLong(1),
                              "sumOfSquares" : NumberDecimal("1")
                      },
                      "keysExamined" : {
                              "sum" : NumberLong(0),
                              "max" : NumberLong(0),
                              "min" : NumberLong(0),
                              "sumOfSquares" : NumberDecimal("0")
                      },
                      "docsExamined" : {
                              "sum" : NumberLong(0),
                              "max" : NumberLong(0),
                              "min" : NumberLong(0),
                              "sumOfSquares" : NumberDecimal("0")
                      },
                      "bytesRead" : {
                              "sum" : NumberLong(0),
                              "max" : NumberLong(0),
                              "min" : NumberLong(0),
                              "sumOfSquares" : NumberDecimal("0")
                      },
                      "readTimeMicros" : {
                              "sum" : NumberLong(0),
                              "max" : NumberLong(0),
                              "min" : NumberLong(0),
                              "sumOfSquares" : NumberDecimal("0")
                      },
                      "workingTimeMillis" : {
                              "sum" : NumberLong(0),
                              "max" : NumberLong(0),
                              "min" : NumberLong(0),
                              "sumOfSquares" : NumberDecimal("0")
                      },
                      "cpuNanos" : {
                              "sum" : NumberLong(914563),
                              "max" : NumberLong(914563),
                              "min" : NumberLong(914563),
                              "sumOfSquares" : NumberDecimal("836425480969")
                      },
                      "delinquentAcquisitions" : {
                              "sum" : NumberLong(0),
                              "max" : NumberLong(0),
                              "min" : NumberLong(0),
                              "sumOfSquares" : NumberDecimal("0")
                      },
                      "totalAcquisitionDelinquencyMillis" : {
                              "sum" : NumberLong(0),
                              "max" : NumberLong(0),
                              "min" : NumberLong(0),
                              "sumOfSquares" : NumberDecimal("0")
                      },
                      "maxAcquisitionDelinquencyMillis" : {
                              "sum" : NumberLong(0),
                              "max" : NumberLong(0),
                              "min" : NumberLong(0),
                              "sumOfSquares" : NumberDecimal("0")
                      },
                      "hasSortStage" : {
                              "true" : NumberLong(0),
                              "false" : NumberLong(1)
                      },
                      "usedDisk" : {
                              "true" : NumberLong(0),
                              "false" : NumberLong(1)
                      },
                      "fromMultiPlanner" : {
                              "true" : NumberLong(0),
                              "false" : NumberLong(1)
                      },
                      "fromPlanCache" : {
                              "true" : NumberLong(0),
                              "false" : NumberLong(1)
                      },
                      "firstSeenTimestamp" : ISODate("2025-08-26T15:11:26.307Z"),
                      "latestSeenTimestamp" : ISODate("2025-08-26T15:11:26.307Z")
              },
              "asOf" : ISODate("2025-08-26T15:11:29.112Z")
      }
      

      to the following:

      {
          key: { /* same */
          },
          keyHash: 'dsoJ+LHAru0z6MJ1/IygJnnLTrlpVYYmPnlmNZbZrLI=',
          queryShapeHash: "uxMLCvpiJ5N/IRqt4c28/0A8F01C8AA16CA805FF5C1A5737535F97E40C2A90CE91A82CCB7A74C7CCB9C48",
          "metrics": {
              "lastExecutionMicros": NumberLong(903),
              "execCount": NumberLong(1),
              "totalExecMicros": {
                  "sum": NumberLong(903),
                  "max": NumberLong(903),
                  "min": NumberLong(903),
                  "sumOfSquares": NumberDecimal("815409")
              },
              "workingTimeMillis": {
                  "sum": NumberLong(0),
                  "max": NumberLong(0),
                  "min": NumberLong(0),
                  "sumOfSquares": NumberDecimal("0")
              },
              "cpuNanos": {
                  "sum": NumberLong(914563),
                  "max": NumberLong(914563),
                  "min": NumberLong(914563),
                  "sumOfSquares": NumberDecimal("836425480969")
              },
              "cursor": {
                  "firstResponseExecMicros": {
                      "sum": NumberLong(903),
                      "max": NumberLong(903),
                      "min": NumberLong(903),
                      "sumOfSquares": NumberDecimal("815409")
                  }
              },
              "queryExec": {
                  "docsReturned": {
                      "sum": NumberLong(1),
                      "max": NumberLong(1),
                      "min": NumberLong(1),
                      "sumOfSquares": NumberDecimal("1")
                  },
                  "keysExamined": {
                      "sum": NumberLong(0),
                      "max": NumberLong(0),
                      "min": NumberLong(0),
                      "sumOfSquares": NumberDecimal("0")
                  },
                  "docsExamined": {
                      "sum": NumberLong(0),
                      "max": NumberLong(0),
                      "min": NumberLong(0),
                      "sumOfSquares": NumberDecimal("0")
                  },
                  "bytesRead": {
                      "sum": NumberLong(0),
                      "max": NumberLong(0),
                      "min": NumberLong(0),
                      "sumOfSquares": NumberDecimal("0")
                  },
                  "readTimeMicros": {
                      "sum": NumberLong(0),
                      "max": NumberLong(0),
                      "min": NumberLong(0),
                      "sumOfSquares": NumberDecimal("0")
                  },
                  "delinquentAcquisitions": {
                      "sum": NumberLong(0),
                      "max": NumberLong(0),
                      "min": NumberLong(0),
                      "sumOfSquares": NumberDecimal("0")
                  },
                  "totalAcquisitionDelinquencyMillis": {
                      "sum": NumberLong(0),
                      "max": NumberLong(0),
                      "min": NumberLong(0),
                      "sumOfSquares": NumberDecimal("0")
                  },
                  "maxAcquisitionDelinquencyMillis": {
                      "sum": NumberLong(0),
                      "max": NumberLong(0),
                      "min": NumberLong(0),
                      "sumOfSquares": NumberDecimal("0")
                  },
              },
              "queryPlanner": {
                  "hasSortStage": {
                      "true": NumberLong(0),
                      "false": NumberLong(1)
                  },
                  "usedDisk": {
                      "true": NumberLong(0),
                      "false": NumberLong(1)
                  },
                  "fromMultiPlanner": {
                      "true": NumberLong(0),
                      "false": NumberLong(1)
                  },
                  "fromPlanCache": {
                      "true": NumberLong(0),
                      "false": NumberLong(1)
                  },
              },
              "firstSeenTimestamp": ISODate("2025-08-26T15:11:26.307Z"),
              "latestSeenTimestamp": ISODate("2025-08-26T15:11:26.307Z")
          },
          "asOf": ISODate("2025-08-26T15:11:29.112Z")
      } 

      I think this would be a breaking change for existing tools such as the data warehouse pipeline and the query stats insights atlas page we're working on. However, I think the workaround is easy. I believe the workaround could be as simple as adding a $project stage to go back to the old format if you want that.

      he motivation here is simply better organization of the metrics. When we began there were fewer than 10. An upcoming project will add 5 or so more, and in the meantime there have already been maybe 8 more. The growing list of top-level fields is hard to understand from a documentation and human perspective.

            Assignee:
            Unassigned
            Reporter:
            Backlog - Core Eng Program Management Team
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: