[SERVER-41243] Allow upsert and delete commands if they can target single shard using query Created: 20/May/19  Updated: 12/Dec/23

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Arun Banala Assignee: Backlog - Cluster Scalability
Resolution: Unresolved Votes: 0
Labels: sharding-common-backlog
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Cluster Scalability
Participants:

 Description   

The current behaviour for upserts and delete commands is that they need to specify full shard key in the query. If not we fail on mongos.

  • This is an unnecessary holdover from the time where we could only target a single shard if we had an exact match on the shard key
  • The real requirement for upsert and non-multi delete is not an exact shard key match, but rather that the operation can target a single shard
  • It is unnecessary to make exact shard key a special requirement, because agnostically targeting the query can resolve to a single shard even with a partial or range query on the shard key, if all relevant chunks are on the same shard
  • Removing this requirement will help the ongoing project to make sharding more transparent, by removing a cluster-only special case that is not enforced for a replica set deployment


 Comments   
Comment by Kevin Pulo [ 21/May/19 ]

While these points are true, meaning this behaviour is technically possible/correct, I don't think it's actually desirable.

The main problem is that the runnability of these ops would depend on the current chunk layout, which is fluid and changes transparently. Ops which depend on unseen state will thus be fragile, and so such dependencies should be avoided.

It's no good being able to run

db.test.update( { _id: { $ge: 1, $le: 3 } }, { ... }, { upsert: true } )

if it fails 5 minutes later because a chunk with range 2 to 4 has been migrated to another shard. In fact it can be worse to allow this to succeed when possible, precisely because there's no guarantee it'll always work. Developers will see these ops work in simple cases, code them into their app, and then be frustrated when one day they suddenly stop working in production for no apparent reason ("nothing changed"). This is the real reason for the exact shard key value match constraint, even though from some angles it looks overly restrictive.

Generated at Thu Feb 08 04:57:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.