[SERVER-21573] "dropDups" option for updates (including findAndModify) Created: 19/Nov/15  Updated: 06/Dec/22  Resolved: 08/May/18

Status: Closed
Project: Core Server
Component/s: Write Ops
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Minor - P4
Reporter: Kevin D. Keck Assignee: Backlog - Query Team (Inactive)
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-544 Merge/overwrite collections function Closed
Assigned Teams:
Query
Participants:

 Description   

It would be convenient to have an update option that would remove the document(s) selected for update (atomically) when the update operation would otherwise produce a duplicate key exception.



 Comments   
Comment by Asya Kamsky [ 08/May/18 ]

Closing this ticket as we have decided that it is not appropriate to have the server drop duplicates during inserts or updates. I.e. our inserts or updates must only affect the one document that they are being applied to and not have a side-effect of removing a different document.

With upcoming support for multi-document transactions, this behavior could be implemented by the application.

Comment by Ramon Fernandez Marina [ 25/Nov/15 ]

Thanks for the detailed example kdkeck. It is unclear whether having the server remove documents is a good idea; at the very least the semantics need to be well defined. Putting this ticket in "Needs Triage" so it can be discussed in the next round of planning.

Comment by Kevin D. Keck [ 19/Nov/15 ]

No, this isn't about insert/upsert, where conceptually the collision is due to a failure of the server to implement the abstraction of serialized operations. In fact I think the new option should be considered incompatible with upsert (i.e., you shouldn't be allowed to set both). Here's an example that may clear things up:

> db.tag.ensureIndex({user:1,object:1,tag:1},{unique:true})
{
	"createdCollectionAutomatically" : true,
	"numIndexesBefore" : 1,
	"numIndexesAfter" : 2,
	"ok" : 1
}
> var uid = new ObjectId();
> db.tag.insert({user:uid, object:34, tag:"star"})
WriteResult({ "nInserted" : 1 })
> db.tag.insert({user:uid, object:35, tag:"star"})
WriteResult({ "nInserted" : 1 })
> db.tag.insert({user:uid, object:35, tag:"spam"})
WriteResult({ "nInserted" : 1 })
> db.tag.find()
{ "_id" : ObjectId("564e599bc04934e40e8716f5"), "user" : ObjectId("564e5994c04934e40e8716f4"), "object" : 34, "tag" : "star" }
{ "_id" : ObjectId("564e599fc04934e40e8716f6"), "user" : ObjectId("564e5994c04934e40e8716f4"), "object" : 35, "tag" : "star" }
{ "_id" : ObjectId("564e59aac04934e40e8716f7"), "user" : ObjectId("564e5994c04934e40e8716f4"), "object" : 35, "tag" : "spam" }
> db.tag.update({object:35}, {$set:{object:34}}, {multi:true})
WriteResult({
	"nMatched" : 0,
	"nUpserted" : 0,
	"nModified" : 0,
	"writeError" : {
		"code" : 11000,
		"errmsg" : "E11000 duplicate key error index: test.tag.$user_1_object_1_tag_1 dup key: { : ObjectId('564e5994c04934e40e8716f4'), : 34.0, : \"star\" }"
	}
})

At this point the application has a bunch of work to do to figure out what operations are going to be needed to achieve the desired result without creating any duplicate keys. Something like:

> db.tag.remove({user:uid, object:35, tag:"star"})
WriteResult({ "nRemoved" : 1 })
> db.tag.update({object:35}, {$set:{object:34}}, {multi:true})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

Imagine instead being able to just do this:

> db.tag.update({object:35}, {$set:{object:34}}, {multi:true, dropDups:true})
WriteResult({ "nMatched" : 2, "nUpserted" : 0, "nModified" : 1, "nRemoved" : 1 })
> db.tag.find()
{ "_id" : ObjectId("564e599bc04934e40e8716f5"), "user" : ObjectId("564e5994c04934e40e8716f4"), "object" : 34, "tag" : "star" }
{ "_id" : ObjectId("564e59aac04934e40e8716f7"), "user" : ObjectId("564e5994c04934e40e8716f4"), "object" : 34, "tag" : "spam" }

Comment by Ramon Fernandez Marina [ 19/Nov/15 ]

kdkeck, is SERVER-14322 the behavior you'd like to see? If not, can you please elaborate on the description above, preferably with a javascript example?

Thanks,
Ramón.

Generated at Thu Feb 08 03:57:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.