[SERVER-68434] Equality to null can incorrectly use a cached partial ixscan which does not cover the predicate Created: 29/Jul/22 Updated: 29/Oct/23 Resolved: 10/Feb/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 6.0.0-rc13 |
| Fix Version/s: | 5.0.15, 7.0.0-rc0, 6.0.5, 4.2.25, 4.4.20 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Nicholas Zolnierz | Assignee: | Nicholas Zolnierz |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | query-director-triage | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||
| Assigned Teams: |
Query Optimization
|
||||||||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||||||||||||||
| Backport Requested: |
v6.0, v5.0, v4.4, v4.2
|
||||||||||||||||||||||||||||||||||||||
| Steps To Reproduce: |
|
||||||||||||||||||||||||||||||||||||||
| Sprint: | QO 2023-02-06, QO 2023-02-20 | ||||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||||
| Linked BF Score: | 16 | ||||||||||||||||||||||||||||||||||||||
| Description |
|
Given an index with a partial filter containing a $or and one branch being {<path>: {$exists: true}}. If a plan cache entry gets created (and is active) for a parameterized query such as {<path>: {$eq: 5}}, then a subsequent query of the form {<path>: {$eq: null}} will incorrectly use the cached plan even though $eq with null does not satisfy the $exists filter since it also matches missing. The $or is significant as I was not able to reproduce this with a single $exists partial index filter. Attached is a minimal repro script. Note that I verified this happens on 6.0 and master, but not on 5.0. |
| Comments |
| Comment by Githook User [ 16/Feb/23 ] | ||
|
Author: {'name': 'Nicholas Zolnierz', 'email': 'nicholas.zolnierz@mongodb.com', 'username': 'nzolnierzmdb'}Message: (cherry picked from commit f15f2bf8958557b4e8fccc6e8e1c7c8c5834d209) | ||
| Comment by Githook User [ 16/Feb/23 ] | ||
|
Author: {'name': 'Nicholas Zolnierz', 'email': 'nicholas.zolnierz@mongodb.com', 'username': 'nzolnierzmdb'}Message: (cherry picked from commit f15f2bf8958557b4e8fccc6e8e1c7c8c5834d209) | ||
| Comment by Githook User [ 14/Feb/23 ] | ||
|
Author: {'name': 'Nicholas Zolnierz', 'email': 'nicholas.zolnierz@mongodb.com', 'username': 'nzolnierzmdb'}Message: (cherry picked from commit f15f2bf8958557b4e8fccc6e8e1c7c8c5834d209) | ||
| Comment by Githook User [ 14/Feb/23 ] | ||
|
Author: {'name': 'Nicholas Zolnierz', 'email': 'nicholas.zolnierz@mongodb.com', 'username': 'nzolnierzmdb'}Message: (cherry picked from commit f15f2bf8958557b4e8fccc6e8e1c7c8c5834d209) | ||
| Comment by Nicholas Zolnierz [ 10/Feb/23 ] | ||
|
The commit description is lacking a ton of details, so I'll attempt to summarize here for future readers. tl;dr The gist of the problem described by this ticket as well as a few of the linked tickets revolves around the way in which the plan cache key encodes the discriminator for partial indexes. If the discriminator is wrong (e.g. <0> even though the filter is eligible to use the index), then there's a chance that a new query of the same shape but different parameters incorrectly uses the cached plan.
against a partial filter expression of
will compare the incoming match expression against each leaf of the $or. The results are AND'ed, and because of the {a: "not five"} expression, the final discriminator is "<0>". This of course is wrong and the query planner can (and likely will) choose to use the partial index.
As for the fix, the approach taken was to move the discriminator for partial index expressions to be "global". The intent is that these discriminators do not act on a certain path; instead they operate against the entire incoming match expression. So for the above case, we will not shred apart the partial filter (nor the incoming query) and will compare the $or holistically to the incoming {a: {$eq: 5}} filter. Since one branch of the $or fully covers the incoming filter, it is safe to say that the query is a strict subset of what is contained in the partial index and the discriminator will be set to "(1)". Note that the "(" and ")" separators are new and as used to designate global discriminators.
| ||
| Comment by Githook User [ 10/Feb/23 ] | ||
|
Author: {'name': 'Nicholas Zolnierz', 'email': 'nicholas.zolnierz@mongodb.com', 'username': 'nzolnierzmdb'}Message: |