Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-86061

In case of EOF optimization and agg pipeline, avoid re-running find query with virtual scan

    • Type: Icon: Improvement Improvement
    • Resolution: Won't Fix
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Query Execution

      In classic_runtime_planner_for_sbe, if the query has an aggregation pipeline, we will multi-plan only find part of the query and later extend the winning solution with agg pipeline, before executing everything in SBE.

      However, in cases, when the winning plan reach EOF, we can avoid re-running find part of the query by replacing it with VirtualScan of returned documents.

      This means the full algorithm will look like this:

      1. Input: a query with aggregation pipeline. Let's denote find part as F and pipeline part as P.
      2. Use multi-planning to find best solution for F.
      3. Extend best solution for F with P and build SBE plan for it. Let's denote this plan as S1.
      4. Put S1 in SBE plan cache for this query.
      5. If best solution reached EOF, build a second QuerySolution: just a VirtualScan of returned documents.
      6. Extend virtual scan solution with P and build SBE plan for it. Let's denote it as S2.
      7. Run S2.

      One performance consideration for this ticket is that we are running sbe stage builders twice: one time for cache, another time for actual execution, which might be pretty expensive process.

      However, given that we are avoiding re-reading at least 101 documents from storage and S2 solution is smaller, this still should be beneficial.

      Results of this optimization should be verified via performance tests.

            Assignee:
            backlog-query-execution [DO NOT USE] Backlog - Query Execution
            Reporter:
            ivan.fefer@mongodb.com Ivan Fefer
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: