[SERVER-80522] [CQF] Correct shard filter documents with missing shard key Created: 29/Aug/23  Updated: 29/Oct/23  Resolved: 31/Aug/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.2.0-rc0

Type: Bug Priority: Major - P3
Reporter: Ben Shteinfeld Assignee: Ben Shteinfeld
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-71169 [CQF] Run existing shard filtering js... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: QO 2023-09-04
Participants:

 Description   

const st = new ShardingTest({
    shards: 1,
});
const dbName = "foo";
const collName = "bar";
const ns = dbName + "." + collName;
assert.commandWorked(st.s.adminCommand({enableSharding: dbName}));
assert.commandWorked(st.s.adminCommand({shardCollection: ns, key: {skey: 1}}));
assert.commandWorked(st.s.getCollection(ns).insert({_id: 1, x: 1}));
assert.sameMembers(st.s.getCollection(ns).find({x: 1}).toArray(), [{_id: 1, x: 1}]);
st.stop(); 

When the following test-case is run under Bonsai, we currently incorrectly omit the document. Inserting some print statements into the code, I observed that the plan which Bonsai generates ends up invoking ShardFilterer::keyBelongsToMe with an empty BSONObj, rather than {skey: null}. Based on the API contract of keyBelongsToMe, this is undefined behavior.

This is confirmed when looking at the plan generated by the SBE stage builders:

shardFilter(s1, makeBsonObj(MakeObjSpec([\"skey\" = Arg(0)], Open), Nothing, (s2 ?: null))) 

The plan produced by Bonsai looks like:

shardFilter(s1, makeBsonObj(MakeObjSpec([\"skey\" = Arg(0)], Open), Nothing, s3)) 



 Comments   
Comment by Githook User [ 31/Aug/23 ]

Author:

{'name': 'Ben Shteinfeld', 'email': 'ben.shteinfeld@mongodb.com', 'username': 'bshteinfeld'}

Message: SERVER-71169 Enable jstests/noPassthrough/catalog_shard.js in CQF variant

The test was failing because we weren't handling missing shard keys properly
until SERVER-80522 was fixed.
Branch: master
https://github.com/mongodb/mongo/commit/db1b9698e0c0d60f07239e1768b9f011909f3607

Comment by Githook User [ 30/Aug/23 ]

Author:

{'name': 'Ben Shteinfeld', 'email': 'ben.shteinfeld@mongodb.com', 'username': 'bshteinfeld'}

Message: SERVER-80522 [CQF] Handle missing shard keys when shard filtering

Components of the shard key can be omitted from user documents. In this
case in the classic engine, the value for those component is clamped to an
explicit `null` when constructing a `ShardKeyPattern`. This is necessary because
the `ShardFilterer` implementation expects all components of the shard key to
have an explicit value. This patch implements this behavior for SBE plans
generated by Bonsai.
Branch: master
https://github.com/mongodb/mongo/commit/ed20dc5bf019ed1ebd0572bb64a78b54621e0f14

Generated at Thu Feb 08 06:43:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.