Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-88692

Consider adding an SBE stage that serves as a materialization point between find and aggregation parts of the plan

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Execution

      If the query contains an aggregation pipeline, multi-planning will be used only for the "find" part of the query. However, we need to cache the whole query, as currently we can't re-combine already prepared SBE plans and we can't recover QuerySolutions from SBE plan cache entries.

      When SBE plan is retrieved from cache, it will go through SBE trial run phase for us to see if replanning is needed.

      In cases where the query was extended with an aggregation pipeline, we need to track the number of documents that reached the aggregation pipeline, because aggregation pipelines can have stages that affect the number of documents unpredictably. For example, blocking stages like $group or stages that produce more documents like $unwind.

      Currently to do that, we are finding a "planning root" - a node that is a root of sub-tree that is responsible for "find" part of the query and attach trial run tracker to it.

      However, because of that, we have to run the whole plan and stash the final results of the plan.

      This may lead to increased memory consumption and early trial run termination.

      To avoid that, we can introduce an extra stage between find and aggregation parts of the query that will do two things:

      1. Track the amount of documents, returned by "find" part.
      2. Can stash the results in it until trial run is over.

      Downside of that is performance. We can do some tricky stuff like this stage will delete itself from the plan once the trial is over, but it will still be always present for queries that return less then 101 results.

            Assignee:
            Unassigned Unassigned
            Reporter:
            ivan.fefer@mongodb.com Ivan Fefer
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated: