[SERVER-84850] Prototype partial pushdown from DocumentSource to PlanStage Created: 08/Aug/19  Updated: 12/Jan/24  Resolved: 15/Aug/19

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: David Storch Assignee: David Storch
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Sprint: Query 2019-08-26
Participants:

 Comments   
Comment by David Storch [ 15/Aug/19 ]

I've completed this POC in patches 4 and 5 of https://mongodbcr.appspot.com/504440019/. The issues at play here are as follows:

  • Should we introduce something like PipelineAstNode? It's helpful to distinguish these nodes from query solution nodes, but in the short term they will get immediately transformed to QuerySolutionNode}}s, and then will get transformed from there to {{PlanStage nodes. Do we need this many node types right now? Or can we get away with just QuerySolutionNode and PlanStage for now?
  • This change results in non-trivial pieces of the execution tree above the MultiPlanStage. This will likely necessitate changes to the explain format, unless we want to bend over backwards to preserve the existing format (which I would be very hesitant to do). Patch 5 of the linked code review demonstrates a simple path forward, which will also help to expose execution stats from any agg stages that get pushed down into the PlanStage execution tree.
  • We'll have to be careful to make sure that we don't break the path for planning from the cache.
  • Planning logic which is built into the getExecutor() layer will probably have to be moved down a layer into the QueryPlanner itself. I suspect that the POC doesn't correctly handle subplanning, logic for generating COUNT_SCAN plans, or logic for generating DISTINCT_SCAN plans.

After patch 5 we can completely eliminate document source execution for pipelines with $unwind, and we can report on what's going on in explain!

CC pawel.terlecki

Generated at Thu Feb 08 06:56:09 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.