Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-86061

In case of EOF optimization and agg pipeline, avoid re-running find query with virtual scan

    XMLWordPrintableJSON

Details

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major - P3 Major - P3
    • None
    • None
    • None
    • None
    • Query Execution

    Description

      In classic_runtime_planner_for_sbe, if the query has an aggregation pipeline, we will multi-plan only find part of the query and later extend the winning solution with agg pipeline, before executing everything in SBE.

      However, in cases, when the winning plan reach EOF, we can avoid re-running find part of the query by replacing it with VirtualScan of returned documents.

      This means the full algorithm will look like this:

      1. Input: a query with aggregation pipeline. Let's denote find part as F and pipeline part as P.
      2. Use multi-planning to find best solution for F.
      3. Extend best solution for F with P and build SBE plan for it. Let's denote this plan as S1.
      4. Put S1 in SBE plan cache for this query.
      5. If best solution reached EOF, build a second QuerySolution: just a VirtualScan of returned documents.
      6. Extend virtual scan solution with P and build SBE plan for it. Let's denote it as S2.
      7. Run S2.

      One performance consideration for this ticket is that we are running sbe stage builders twice: one time for cache, another time for actual execution, which might be pretty expensive process.

      However, given that are are avoiding re-reading 101 documents from storage and S2 solution is smaller, this still should be beneficial.

      Results of this optimization should be verified via performance tests.

      Attachments

        Activity

          People

            backlog-query-execution Backlog - Query Execution
            ivan.fefer@mongodb.com Ivan Fefer
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: