[SERVER-82982] [CQF] jstests/cqf/optimizer/sort_merge.js fails with parameterization enabled Created: 08/Nov/23 Updated: 24/Jan/24 |
|
| Status: | Open |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Ben Shteinfeld | Assignee: | Backlog - Query Optimization |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Query Optimization
|
||||||||
| Operating System: | ALL | ||||||||
| Sprint: | QO 2023-11-13, QO 2023-11-27 | ||||||||
| Participants: | |||||||||
| Description |
|
This test fails when parameterization is enabled with a tassert that the optimizer fails to generate a plan. The query is
The predicate is translated as
When we try to convert this ABT to PartialSchemaRequirements, we fail because we only support building bounds for EqMember when the operand is a Constant. This results in us failing to convert this filter to a SargableNode, which is necessary for the optimizer to ever emit an ixscan plan. With the hint, we disable collscan plans, so we fail to generate a plan. |
| Comments |
| Comment by Ben Shteinfeld [ 10/Nov/23 ] | |||||||||||||||||
|
If we are willing to parameterize each element of the $in operand separately, I believe the existing code would handle this case without changes. The disadvantage of this is potentially a large number of parameters for a query. Another approach would be to parameterize the entire operand of $in and generate the PSR using the getElement SBE function for each index of the array. For example, {$in: [1, 2]} would be parameterized as a single parameter (getParam(0)). The intervals we could generate are [getElement(getParam(0), 0), getElement(getParam(0), 0)] U [getElement(getParam(0), 1), getElement(getParam(0), 1)]. | |||||||||||||||||
| Comment by Ben Shteinfeld [ 09/Nov/23 ] | |||||||||||||||||
|
We are currently unable to generate an ixscan plan for a $in with a parameterized operand. The example above just shows a test failure where the hint expects us to generate an ixscan plan and we fail to do so. Looking at how SBE stage builders handle this case. Consider a query find({a: {$in: [0, 10, 20, 30]}}),
The SBE stage builders use IETs to generate a set of index bounds that are bound into s11 in the runtime environment (global slots). These bounds are exposed to the plan using unwind, project on the outer side of an NLJ, where the inner side is an ixscan. We need to have a larger design discussion about we expect parameterized ixscan plans to work in Bonsai. The decisions we make there will largely inform how we choose to handle this specific case with $in. I don't know whether we will handle this as part of this project, or a follow up which deals specifically with plan cache for M4. I think in the short term, we should disable $in parameterization or only perform it when there are no indexes (M2 eligible queries). |