[SERVER-75108] Revisit implementation of explain for $unionWith Created: 21/Mar/23  Updated: 02/Aug/23

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: David Storch Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-50597 Explain against a pipeline containing... Backlog
is related to SERVER-71636 Explain executionStats does not work ... Closed
Assigned Teams:
Query Optimization
Participants:

 Description   

Before $unionWith was implemented, all DocumentSource derived classes followed the same implementation pattern for "executionStats" or ""allPlansExecution" verbosity explain operations. Namely, the explain is done in two phases:

  • First, the pipeline is fully executed and the resulting data discarded. That happens here. During execution, runtime stats are captured and stored inside the stages of the pipeline.
  • Second, Pipeline::writeExplainOps() is called. This is intended to serialize the pipeline and its runtime stats without actually doing any further query execution.

$unionWith, however, does not work as described above. Rather than serializing its original subpipeline during DocumentSourceUnionWith::serialize() it issues an entirely new explain operation related to the subpipeline which may do further query execution. This goes against the typical design for aggregation explain and requires some special infrastructure in the MongoProcessInterface. It may have some benefit in that we are able to provide more detailed explain output in the case that the collection on the inner side is sharded. However, it seems worth at least looking into whether we can reimplement explain for $unionWith to avoid this complexity.



 Comments   
Comment by David Storch [ 21/Mar/23 ]

I'm not sure whether this belongs with Query Execution or Query Optimization, but starting with QO.

CC hana.pearlman@mongodb.com ted.tuckman@mongodb.com

Generated at Thu Feb 08 06:29:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.