[SERVER-44422] Allow findAndModify and delete one to target by query instead of extracted shard key Created: 05/Nov/19  Updated: 29/Oct/23  Resolved: 12/Jul/23

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 7.1.0-rc0, 5.0.20, 6.0.9, 7.0.2

Type: Improvement Priority: Major - P3
Reporter: Jack Mulrow Assignee: Wenqin Ye
Resolution: Fixed Votes: 1
Labels: ShardingRoughEdges
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-78385 Update findAndModify and deleteOne to... Closed
Documented
is documented by DOCS-16308 Investigate changes in SERVER-44422: ... Closed
Related
related to SERVER-61683 Operation findAndModify not possible ... Closed
related to SERVER-80307 Allow findAndModify to update shard k... Closed
related to SERVER-78894 Allow update to work with a partial s... Closed
Assigned Teams:
Sharding NYC
Backwards Compatibility: Fully Compatible
Backport Requested:
v7.0, v6.0, v5.0
Sprint: Sharding NYC 2023-06-26, Sharding NYC 2023-07-10, Sharding NYC 2023-07-24
Participants:
Case:

 Description   

findAndModify and delete with justOne=true must be targeted to a single shard which they check by extracting a shard key from their query and targeting the chunk that owns that key. This is done through ShardKeyPattern::extractShardKeyFromQuery() which requires a simple query that will exactly match some value, e.g. {skey: 5} or {skey; {$eq: 5}}. More complex queries that don't produce an exact match, but can still be targeted to a single shard are currently rejected.

This could be improved by changing findAndModify and delete to target using ChunkManager::getShardIdsForQuery(), which returns the shards that own ranges that overlap with the shard key index bounds generated by the query (after checking for an exact shard key match first), and throwing if more than one shard is targeted. This is already how an update is targeted by its query .



 Comments   
Comment by Githook User [ 20/Sep/23 ]

Author:

{'name': 'Wenqin Ye', 'email': 'wenqin908@gmail.com', 'username': 'wenqinYe'}

Message: SERVER-81211: Fix failing asserts due to backporting of SERVER-44422 to 7.0
Branch: master
https://github.com/mongodb/mongo/commit/2afcdb651ade420bba3f61611e946457148993a3

Comment by Githook User [ 20/Sep/23 ]

Author:

{'name': 'Wenqin Ye', 'email': 'wenqin908@gmail.com', 'username': 'wenqinYe'}

Message: SERVER-81210: Fix failing asserts on 7.1 due to backporting of SERVER-44422 to 7.0
Branch: v7.1
https://github.com/mongodb/mongo/commit/d0ab228bcf343c0c17c8db3dc92a9448a2c18034

Comment by Githook User [ 18/Sep/23 ]

Author:

{'name': 'wenqinYe', 'email': 'wenqin908@gmail.com', 'username': 'wenqinYe'}

Message: SERVER-44422: [v7.0 backport] Allow findAndModify and delete one to target by query instead of extracted shard key
Branch: v7.0
https://github.com/mongodb/mongo/commit/009489e71bbce357b53259d81dc909c7638683e9

Comment by Githook User [ 02/Aug/23 ]

Author:

{'name': 'wenqinYe', 'email': 'wenqin908@gmail.com', 'username': 'wenqinYe'}

Message: SERVER-44422: (6.0 Backport) Allow findAndModify and delete one to target by query instead of extracted shard key
Branch: v6.0
https://github.com/mongodb/mongo/commit/678953a5f1d3803087d06465743b0fe1d2260c3b

Comment by Githook User [ 02/Aug/23 ]

Author:

{'name': 'wenqinYe', 'email': 'wenqin908@gmail.com', 'username': 'wenqinYe'}

Message: SERVER-44422: (5.0 Backport) Allow findAndModify and delete one to target by query instead of extracted shard key
Branch: v5.0
https://github.com/mongodb/mongo/commit/985b8ab12aaa0b11af8f7eec7450ec62b28e4ca4

Comment by Githook User [ 12/Jul/23 ]

Author:

{'name': 'wenqinYe', 'email': 'wenqin908@gmail.com', 'username': 'wenqinYe'}

Message: SERVER-44422: Allow findAndModify and delete one to target by query instead of extracted shard key
Branch: master
https://github.com/mongodb/mongo/commit/cbd1d47237bcff430c15840d3d3f97d85d4a0b0a

Comment by Jack Mulrow [ 06/Nov/19 ]

kaloian.manassiev, yes I'd say so. Previously a user would never want to use predicates like that though, since all documents needed to have the full shard key. Now that we've lifted that requirement, this might be a useful feature going forward.

Comment by Kaloian Manassiev [ 06/Nov/19 ]

jack.mulrow, based on our discussion yesterday, since findAndModify never supported predicates which did not contain the complete shard key, this is technically not a regression, right?

Comment by Jack Mulrow [ 05/Nov/19 ]

This is a little more important after SERVER-42390 because we allowed missing shard key fields in user documents, and a user could query for such documents using {skey: {$exists: false}}, but that operator isn't considered an exact equality match, so we can't use it in the commands that need to target a single shard.

Generated at Thu Feb 08 05:05:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.