-
Type: Improvement
-
Resolution: Unresolved
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Query Execution
If the query contains an aggregation pipeline, multi-planning will be used only for the "find" part of the query. However, we need to cache the whole query, as currently we can't re-combine already prepared SBE plans and we can't recover QuerySolutions from SBE plan cache entries.
When SBE plan is retrieved from cache, it will go through SBE trial run phase for us to see if replanning is needed.
In cases where the query was extended with an aggregation pipeline, we need to track the number of documents that reached the aggregation pipeline, because aggregation pipelines can have stages that affect the number of documents unpredictably. For example, blocking stages like $group or stages that produce more documents like $unwind.
Currently to do that, we are finding a "planning root" - a node that is a root of sub-tree that is responsible for "find" part of the query and attach trial run tracker to it.
However, because of that, we have to run the whole plan and stash the final results of the plan.
This may lead to increased memory consumption and early trial run termination.
To avoid that, we can introduce an extra stage between find and aggregation parts of the query that will do two things:
- Track the amount of documents, returned by "find" part.
- Can stash the results in it until trial run is over.
Downside of that is performance. We can do some tricky stuff like this stage will delete itself from the plan once the trial is over, but it will still be always present for queries that return less then 101 results.
- is related to
-
SERVER-88488 Improve RAM consumption for trial runs of cached plans in SBE
- Closed