[SERVER-78752] An incorrect plan can be written to the SBE plan cache for certain $or queries, leads to incorrect query results Created: 06/Jul/23 Updated: 29/Oct/23 Resolved: 20/Jul/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 6.3.1, 7.0.0-rc7 |
| Fix Version/s: | 7.1.0-rc0, 7.0.0-rc8 |
| Type: | Bug | Priority: | Blocker - P1 |
| Reporter: | Reginald Chounoune | Assignee: | David Storch |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | auto-reverted | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||||||||||
| Issue Links: |
|
||||||||||||||||||||||||
| Assigned Teams: |
Query Execution
|
||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||
| Sprint: | QE 2023-07-24 | ||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||
| Linked BF Score: | 167 | ||||||||||||||||||||||||
| Description |
|
After analysis the following root cause is two fold: 1) The Parameterization of a query abstracts dependencies between two $or branches in such a way that dependencies are lost. A query containing an $or statement with predicates on the same field of the same value are considered to be the same as a query where the 2 filters did not have the same value. Both queries would result in the same QueryShape which is semantically incorrect as the dependency are used in query optimizations. These two queries should have two different planhashes 2) While checking for equality of 2 $or branches containing and IndexScan the equality function was missing to check the the IETs and 2 falsely considered equal IndexScans were merged together leading to a correctness issue. Hi Team, I have identified an issue with the Slot-based Execution Engine where the Query Planner is selecting the wrong plan for a find() command (see details below), resulting in data inconsistency. In summary, it's implicitly adding a limit(1) to a query, causing the query to always return 1 document instead of the actual number of matched documents (2 expected). Before we found the actual culprit, one workaround was to do a test failover in Atlas also which would resolve the issue temporarily. Scenario:
I have also attached the Plan cache output which contains the bad plans along with the new plans (after we cleared the cache). Each section is labeled "old cache query plans" and "new cached query plans" respectively. Please let me know if you need any additional information. Thank you for your help. Best regards, |
| Comments |
| Comment by Githook User [ 20/Jul/23 ] |
|
Author: {'name': 'David Storch', 'email': 'david.storch@mongodb.com', 'username': 'dstorch'}Message: This reintroduces reverted commit |
| Comment by Githook User [ 20/Jul/23 ] |
|
Author: {'name': 'David Storch', 'email': 'david.storch@mongodb.com', 'username': 'dstorch'}Message: |
| Comment by Githook User [ 19/Jul/23 ] |
|
Author: {'name': 'David Storch', 'email': 'david.storch@mongodb.com', 'username': 'dstorch'}Message: |
| Comment by xgen-buildbaron-user [ 19/Jul/23 ] |
|
Ticket re-opened due to revert. replica_sets_terminate_primary_jscore_passthrough began a consistent failure of jstests/core/sbe_plan_cache_duplicate_or_clauses.js |
| Comment by Githook User [ 19/Jul/23 ] |
|
Author: {'name': 'auto-revert-processor', 'email': 'dev-prod-dag@mongodb.com', 'username': ''}Message: Revert " This reverts commit b1453cb73fbdc3b849dd8b9761c6c1b255452e2e. |
| Comment by Githook User [ 19/Jul/23 ] |
|
Author: {'name': 'Peter Volk', 'email': 'peter.volk@mongodb.com', 'username': 'HCSPete'}Message: (cherry picked from commit b1453cb73fbdc3b849dd8b9761c6c1b255452e2e) |
| Comment by Githook User [ 19/Jul/23 ] |
|
Author: {'name': 'Peter Volk', 'email': 'peter.volk@mongodb.com', 'username': 'HCSPete'}Message: |