-
Type: Improvement
-
Resolution: Fixed
-
Priority: Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
-
Query Execution
-
Fully Compatible
-
QE 2024-04-01
In classic runtime planner for SBE when running a trial period for cached plan, we track the amount of documents that are returned by find part of the query, not the whole pipeline.
Consider the pipeline:
[{$match: ..}, {$lookup: ..}, {$unwind: ..}]
When first we multi-plan the query, we will run only $match part.
After that we will extend the plan with the pipeline and run it.
In SBE plan cache, we will only have the whole plan.
When recovering a plan from cache, we want to run a trial period to make sure plan is still effective enough. So we set a read budget and run the plan until it returns enough documents.
We can't just look at the amount of documents, returned by the whole plan, as $unwind may affect it randomly.
So we track via TrialRunTracker the amount of documents, returned by the part of the plan that correspond to $match stage only.
However, we can't run only a part of the plan, so we run the whole plan.
And when the whole plan returns a document, we need to stash it.
However, $lookup + $unwind can inflate the amount of memory needed for this stash.
Currently we just cap the stash at 101 * max document size.
We want to come up with more memory efficient way to do with.
- is depended on by
-
SERVER-83344 Enable Classic multi-planner for SBE feature flag
- Closed
- related to
-
SERVER-88692 Consider adding an SBE stage that serves as a materialization point between find and aggregation parts of the plan
- Needs Scheduling