[SERVER-65085] SBE plan cache entries can be incorrectly reused after a refineCollectionShardKey operation Created: 30/Mar/22 Updated: 29/Oct/23 Resolved: 27/Apr/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Query Planning, Sharding |
| Affects Version/s: | None |
| Fix Version/s: | 6.0.0-rc3, 6.1.0-rc0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | David Storch | Assignee: | Denis Grebennicov |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||
| Backport Requested: |
v6.0
|
||||||||||||||||||||
| Sprint: | QE 2022-04-04, QE 2022-04-18, QE 2022-05-02 | ||||||||||||||||||||
| Participants: | |||||||||||||||||||||
| Linked BF Score: | 146 | ||||||||||||||||||||
| Description |
|
An SBE plan cache entry which was created prior to the shard key being refined can be re-used after the shard key refinement completes. The result is that a query can use an incorrect plan. The plan will be doing orphan filtering incorrectly, leading to incorrect results. The reason that SBE plans cannot be reused across shard key refinement boundaries relates to how orphan filtering is implemented in SBE. Here is an example SBE plan doing orphan filtering for a shard key of {a: 1}:
This plan boils down to a collection scan followed by shard filtering. Importantly, the plan explicitly encodes the steps necessary to extract the shard key and insert it into slot s13:
Let's say that the shard key is refined to something like {a: 1, b: 1}. This plan is no longer valid, because it only extracts values for the "a" field of the shard key, and the "b" part will be completely ignored. The result is that incorrect shard keys are passed to the "shardFilter()" builtin function, and orphan filtering is done incorrectly. |
| Comments |
| Comment by Githook User [ 27/Apr/22 ] |
|
Author: {'name': 'Denis Grebennicov', 'email': 'denis.grebennicov@mongodb.com', 'username': 'denis631'}Message: |
| Comment by Githook User [ 27/Apr/22 ] |
|
Author: {'name': 'Denis Grebennicov', 'email': 'denis.grebennicov@mongodb.com', 'username': 'denis631'}Message: |
| Comment by David Storch [ 12/Apr/22 ] |
|
After discussing with anton.korshunov@mongodb.com and the sharding team, the current plan is for this ticket to encode the shard version epoch and timestamp into the plan cache key. We will not do anything under this ticket to try to clean up cache entries associated with the old epoch when a shard key refinement completes. I filed a followup ticket under which we can look into implementing this more timely plan cache invalidation: CC max.hirschhorn@mongodb.com tommaso.tocci@mongodb.com kaloian.manassiev@mongodb.com |