[DOCS-14453] Investigate changes in SERVER-53762: Report aggregate execution stats in explain for the inner side of $lookup Created: 13/May/21  Updated: 13/Nov/23  Resolved: 23/Aug/21

Status: Closed
Project: Documentation
Component/s: manual, Server
Affects Version/s: None
Fix Version/s: 5.0.0-rc0, Server_Docs_20231030, Server_Docs_20231106, Server_Docs_20231105, Server_Docs_20231113

Type: Task Priority: Major - P3
Reporter: Backlog - Core Eng Program Management Team Assignee: Jason Price
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
documents SERVER-53762 Report aggregate execution stats in e... Closed
Participants:
Days since reply: 2 years, 24 weeks, 2 days ago
Epic Link: DOCSP-15042
Story Points: 3

 Description   

Description

Downstream Change Summary

In this ticket, we've modified the explain output for aggregations using $lookup. If the explain verbosity is allPlansExecution or above, we will add total aggregate execution stats to the output. (The specific stats are totalDocsExamined, totalKeysExamined, collectionScans (the number as an integer), and indexesUsed (an array of the index names).

Description of Linked Ticket

As part of SERVER-47640, we enhanced the implementation of $lookup to keep track of the PlanSummaryStats associated with the subpipeline. These stats include metrics such as the number of keys examined and the number of documents examined by the subpipeline. Note that the subpipeline may be executed many times, since it is typically executed once per document from the outer (or "local") side. The PlanSummaryStats are tracked in aggregate across all such executions of the subpipeline. So if, for instance, the subpipeline scans a 10 document collection every time it runs, and it runs 5 times, then the result will be a "docsExamined" quantity of 50.

Now that we are tracking these values, it would be valuable for debugging/diagnostics to expose them in "executionStats" or "allPlansExecution" level explain output. We will need to choose which metrics are exposed, but I think it would at the very least include docs examined and keys examined. This would be especially helpful in understanding the performance properties of the subpipeline, since explain currently does not display the execution plan used on the inner side (see SERVER-22622). Any information in explain as to whether the $lookup subpipeline is cheap or expensive and why would be immensely helpful for query performance debugging scenarios.

Scope of changes

Consider if we need to update the $lookup docs as part of this change.

Impact to Other Docs

MVP (Work and Date)

Resources (Scope or Design Docs, Invision, etc.)



 Comments   
Comment by Githook User [ 23/Aug/21 ]

Author:

{'name': 'jason-price-mongodb', 'email': 'jshfjghsdfgjsdjh@aolsdjfhkjsdhfkjsdf.com'}

Message: DOCS-14453 execution stats inner lookup

DOCS-14453 execution stats inner lookup
Branch: master
https://github.com/mongodb/docs/commit/872ec941df29fd4d0d2684b6fbbed8cf25d0bec5

Generated at Thu Feb 08 08:10:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.