Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-32014

scanAndOrder serverStatus metric should not be incremented for $sort stages that are merging pre-sorted results

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Duplicate
    • Affects Version/s: 3.4.0, 3.6.0
    • Fix Version/s: None
    • Component/s: Aggregation Framework
    • Labels:
    • Operating System:
      ALL
    • Backport Requested:
      v3.6, v3.4
    • Case:

      Description

      An aggregation pipeline computes whether or not it has a sort stage by scanning through the pipeline looking for a DocumentSourceSort. This will include any $sort stages that are simply merging the pre-sorted results of multiple shards. Those stages should not count towards the 'hasSortStage' metric, since they will not actually perform a blocking sorting algorithm, which is what that metric is intended to represent. This also impacts the serverStatus metric "metrics.operation.scanAndOrder".

      Original description, shamelessly copied from report by Akira Kurogane:

      The following agg command would cause a double-increment of serverStatus.metrics.operation.scanAndOrder on shards that it was targeted to, and one more increment again if it was the shard that the merge step happened on.

      db.coll.aggregate([
        {$match: {A, B, C}}, //i.e. potentially multi-shard
        {$sort: {C, D}} //i.e. something not index-compatible
      ])
      

      It looks to [Akira] this happens because recordCurOpMetrics() will increment that scanAndOrder once for every aggregation stage that has hasSortStage == true, and the PipelineCommand sets sets that property too when it finds any child aggregation stage is a DocumentSourceSort.
      Lastly there is a third increment on the shard the merge happens. [Akira] guesses that is the ClusterAggregate command also setting the hasSortStage property true.

      We should ensure that any aggregation operation will only increment the scanAndOrder metric at most once on each shard.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              backlog-server-query Backlog - Query Team
              Reporter:
              charlie.swanson Charlie Swanson
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: