[SERVER-65085] SBE plan cache entries can be incorrectly reused after a refineCollectionShardKey operation Created: 30/Mar/22  Updated: 29/Oct/23  Resolved: 27/Apr/22

Status: Closed
Project: Core Server
Component/s: Query Planning, Sharding
Affects Version/s: None
Fix Version/s: 6.0.0-rc3, 6.1.0-rc0

Type: Bug Priority: Major - P3
Reporter: David Storch Assignee: Denis Grebennicov
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-64315 Re-enable caching of single solution ... Closed
Related
related to SERVER-65492 Invalidate plan cache entries when th... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.0
Sprint: QE 2022-04-04, QE 2022-04-18, QE 2022-05-02
Participants:
Linked BF Score: 146

 Description   

An SBE plan cache entry which was created prior to the shard key being refined can be re-used after the shard key refinement completes. The result is that a query can use an incorrect plan. The plan will be doing orphan filtering incorrectly, leading to incorrect results.

The reason that SBE plans cannot be reused across shard key refinement boundaries relates to how orphan filtering is implemented in SBE. Here is an example SBE plan doing orphan filtering for a shard key of {a: 1}:

[2] filter {shardFilter (s5, s13)}
[2] project [s13 = if (! exists (s11), Nothing, s12)]
[2] mkbson s12 [a = s11] true false
[2] project [s11 = let [l1.0 = fillEmpty (getField (s6, \"a\"), null)] if (isArray (l1.0), Nothing, l1.0)]
[1] filter {fillEmpty (s10, false)}
[1] traverse s10 s9 s8 [s6, s7] {s10 || s9} {s10}
from
    [1] project [s8 = getField (s6, \"tid\")]
    [1] scan s6 s7 none none none none [] @\"92b69019-c6fd-4bb6-a3b7-511e0b8517b2\" true false
in
    [1] project [s9 = fillEmpty (s8 == 3, false)]
    [1] limit 1
    [1] coscan

This plan boils down to a collection scan followed by shard filtering. Importantly, the plan explicitly encodes the steps necessary to extract the shard key and insert it into slot s13:

[2] project [s13 = if (! exists (s11), Nothing, s12)]
[2] mkbson s12 [a = s11] true false
[2] project [s11 = let [l1.0 = fillEmpty (getField (s6, \"a\"), null)] if (isArray (l1.0), Nothing, l1.0)]

Let's say that the shard key is refined to something like {a: 1, b: 1}. This plan is no longer valid, because it only extracts values for the "a" field of the shard key, and the "b" part will be completely ignored. The result is that incorrect shard keys are passed to the "shardFilter()" builtin function, and orphan filtering is done incorrectly.



 Comments   
Comment by Githook User [ 27/Apr/22 ]

Author:

{'name': 'Denis Grebennicov', 'email': 'denis.grebennicov@mongodb.com', 'username': 'denis631'}

Message: SERVER-65085 SBE plan cache entries can be incorrectly reused after a refineCollectionShardKey operation
Branch: v6.0
https://github.com/mongodb/mongo/commit/f1bdce4f8eae69d14237f76edd1f8511b1f78d98

Comment by Githook User [ 27/Apr/22 ]

Author:

{'name': 'Denis Grebennicov', 'email': 'denis.grebennicov@mongodb.com', 'username': 'denis631'}

Message: SERVER-65085 SBE plan cache entries can be incorrectly reused after a refineCollectionShardKey operation
Branch: master
https://github.com/mongodb/mongo/commit/a43a75a97c7b25adbbdb176560afe1cffbac4aa6

Comment by David Storch [ 12/Apr/22 ]

After discussing with anton.korshunov@mongodb.com and the sharding team, the current plan is for this ticket to encode the shard version epoch and timestamp into the plan cache key. We will not do anything under this ticket to try to clean up cache entries associated with the old epoch when a shard key refinement completes. I filed a followup ticket under which we can look into implementing this more timely plan cache invalidation: SERVER-64315.

CC max.hirschhorn@mongodb.com tommaso.tocci@mongodb.com kaloian.manassiev@mongodb.com

Generated at Thu Feb 08 06:01:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.