-
Type:
Improvement
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Query Optimization
-
135
-
None
-
None
-
None
-
None
-
None
-
None
-
None
This is about reverting SERVER-89382, which disabled $match pushdown past $replaceRoot/$replaceWith when the $match contains $expr. The exception is a workload-specific workaround that does not generalise and blocks a class of other rewrites.
Background
We allowed $match pushdown past $replaceRoot when the new root unnests a path in SERVER-88220:
{$replaceWith: "$subDocument"}, {$match: {x: 2}}
| | |
v v v
{$match: {"subDocument.x": 2}}, {$replaceWith: "$subDocument"}
SERVER-89382 then disabled this pushdown whenever the $match contains $expr, in response to BF-32590: a ~37% regression on TPCHDenormalizedQuery15.
That Q15 workload runs the $expr predicate over a field l_shipdate which is constructed from sequence of $unwind/$addFields/$unwind/$replaceRoot, so it likely canot use an index. Pushing the $expr predicate before the $replaceWith moved it onto a longer dotted path ($lineitem.l_shipdate) on a larger document (pre-unnest) and added a type check. This increased the execution time and produced no offsetting filtering benefit.
Why we should reconsider SERVER-89382
- The workaround is inconsistent with the rest of the optimiser. A $project/$set rename with a dotted path is semantically the same situation:
{$project: {a: "$b.c"}}, {$match: {$expr: {$eq: ["$a", 1]}}} | | | v v v {$match: {$expr: {$eq: ["$b.c", 1]}}}, {$project: {a: "$b.c"}}Here we do not prevent the rewrite even though the $expr has a deeper path and could have similar limitations.
- The special-cased workload is the one case with no beneficial rewrite. But the block forbids the high-value cases, where the predicate can either be used for IXSCAN or re-ordered before other operations. When the unnested path maps to a real indexed field on the base collection, pushing the $expr predicate down could enable an IXSCAN instead of a COLLSCAN. Blocking all $expr sacrifices a potentially large win (IXSCAN vs COLLSCAN) to avoid a regression on an unindexed synthetic workload.
Proposal
Accept the BF-32590 regression on the unindexed TPC-H Q15 as the cost of the alignment, and add a targeted test demonstrating the index-enabled (IXSCAN) case the block currently forbids.
We know that a $project: {a: "$b.c"} rewrite for $match with $expr is useful and do not guard that case, so it seems there isn't a very good reason to guard $replaceRoot in an equivalent case (which produces a deeper path).
- is related to
-
SERVER-89382 Disallow match pushdown for replaceRoot if match contains $expr
-
- Closed
-
-
SERVER-88220 Enable match pushdown for replaceRoot
-
- Closed
-