[SERVER-48380] Expose total data size in bytes processed by $sort and $group in agg execution stats explain Created: 21/May/20  Updated: 29/Oct/23  Resolved: 14/Dec/20

Status: Closed
Project: Core Server
Component/s: Aggregation Framework, Diagnostics
Affects Version/s: None
Fix Version/s: 4.9.0

Type: Improvement Priority: Major - P3
Reporter: David Storch Assignee: Rishab Joshi (Inactive)
Resolution: Fixed Votes: 0
Labels: qexec-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Documented
is documented by DOCS-14454 Investigate changes in SERVER-48380: ... Closed
Related
related to SERVER-49027 Extend $group explain execution stats... Closed
related to SERVER-53303 Make sort and group execution stages ... Closed
is related to SERVER-53461 totalDataSizeGroupedBytesEstimate sta... Closed
is related to SERVER-21784 Track 'nReturned' and 'executionTimeM... Closed
is related to SERVER-48378 Expose settings of query knobs in agg... Closed
Backwards Compatibility: Minor Change
Sprint: Query 2020-11-30, Query 2020-12-14
Participants:

 Description   

SERVER-21784 recently added execution stats to the agg execution layer, and exposed them via "executionStats" or "allPlansExecution" explain verbosities. This ticket, however, added only nReturned and executionTimeMillis for every stage. There are more stats that we can expose which will be useful for debugging and performance investigations.

One suggestion from alex.bevilacqua is to expose the amount of data processed by $sort or $group. We have such stats for sorts executed in the PlanStage layer, but not for sorts executed in the DocumentSource layer. The $sort stage would report a totalDataSizeSortedBytesEstimate metric, and the $group stage would report totalOutputDataSizeBytes.

Another idea that we could consider implementing at the same time is to report usedDisk:true when either a $sort or a $group spills to disk at runtime.


Generated at Thu Feb 08 05:16:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.