Investigate changes in SERVER-99642: Unify spilling statistics in explain output

XMLWordPrintableJSON

    • Type: Investigation
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • None
    • Developer Tools

      Original Downstream Change Summary

      The `executionStats` and `allPlanExecution` explain output for stages that can spill now include the following fields:
      spills: The number of times the stage spilled to disk.
      spilledBytes: The total amount of data spilled to disk in bytes.
      spilledRecords: The total number of records spilled to disk.
      spilledDataStorageSize: The total disk space used by spilled data in bytes.
      The previous fields that reported this information have been replaced by these new fields. Specifically:
      `spilledBytesApprox` is now `spilledBytes` in `$lookup` and `$lookup-unwind` stages.
      `numBytesSpilledEstimate` is now `spilledBytes` in `$group` and `$setWindowFields` stages.

      Description of Linked Ticket

      Currently a lot of stages can spill to disk, but in explain output they will report different sub-set of spilling statistics under different names:

      DocumentSourceGroup will report spilled bytes as numBytesSpilledEstimate

      DocumentSourceGraphLookup will report it as spilledBytes.

      DocumentSourceSort won't even report this metric

      In SERVER-99564 we will ensure that all spilling stages track internally all relevant metrics consistently.

      In this ticket we should make the change to explain output to be consistent as well.

            Assignee:
            Unassigned
            Reporter:
            Backlog - Core Eng Program Management Team
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: