Investigate allowing $expr pushdown past $replaceRoot

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Optimization
    • 135
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      This is about reverting SERVER-89382, which disabled $match pushdown past $replaceRoot/$replaceWith when the $match contains $expr. The exception is a workload-specific workaround that does not generalise and blocks a class of other rewrites.

      Background

      We allowed $match pushdown past $replaceRoot when the new root unnests a path in SERVER-88220:

      {$replaceWith: "$subDocument"}, {$match: {x: 2}}
      | | |
      v v v
      {$match: {"subDocument.x": 2}}, {$replaceWith: "$subDocument"} 

      SERVER-89382 then disabled this pushdown whenever the $match contains $expr, in response to BF-32590: a ~37% regression on TPCHDenormalizedQuery15.

      That Q15 workload runs the $expr predicate over a field l_shipdate which is constructed from sequence of $unwind/$addFields/$unwind/$replaceRoot, so it likely canot use an index. Pushing the $expr predicate before the $replaceWith moved it onto a longer dotted path ($lineitem.l_shipdate) on a larger document (pre-unnest) and added a type check. This increased the execution time and produced no offsetting filtering benefit.

      Why we should reconsider SERVER-89382

      1. The workaround is inconsistent with the rest of the optimiser. A $project/$set rename with a dotted path is semantically the same situation: 
        {$project: {a: "$b.c"}}, {$match: {$expr: {$eq: ["$a", 1]}}}
        | | |
        v v v
        {$match: {$expr: {$eq: ["$b.c", 1]}}}, {$project: {a: "$b.c"}}

        Here we do not prevent the rewrite even though the $expr has a deeper path and could have similar limitations.

      1. The special-cased workload is the one case with no beneficial rewrite. But the block forbids the high-value cases, where the predicate can either be used for IXSCAN or re-ordered before other operations. When the unnested path maps to a real indexed field on the base collection, pushing the $expr predicate down could enable an IXSCAN instead of a COLLSCAN. Blocking all $expr sacrifices a potentially large win (IXSCAN vs COLLSCAN) to avoid a regression on an unindexed synthetic workload.

      Proposal

      Accept the BF-32590 regression on the unindexed TPC-H Q15 as the cost of the alignment, and add a targeted test demonstrating the index-enabled (IXSCAN) case the block currently forbids.

      We know that a $project: {a: "$b.c"} rewrite for $match with $expr is useful and do not guard that case, so it seems there isn't a very good reason to guard $replaceRoot in an equivalent case (which produces a deeper path).

            Assignee:
            Unassigned
            Reporter:
            Vesko Karaganev
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: