[SERVER-10711] $setOnInsert mods should not conflict with non-insert modifiers Created: 09/Sep/13  Updated: 06/Dec/22  Resolved: 29/Jun/19

Status: Closed
Project: Core Server
Component/s: Write Ops
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: kic Assignee: Backlog - Query Team (Inactive)
Resolution: Won't Do Votes: 46
Labels: insert, upsert
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-14912 Need alternative upsert for time seri... Closed
is duplicated by SERVER-15801 Cannot $inc positional array while pe... Closed
Related
related to SERVER-13578 add $setOnUpdate to update operation Closed
related to SERVER-6399 Refactor update() code Closed
is related to SERVER-2643 Allow Field Name Duplication with Mod... Closed
is related to SERVER-30587 Doing Upsert in Subdocuments when per... Closed
Assigned Teams:
Query
Operating System: ALL
Participants:

 Description   

This command results in a conflict:

db.test.update({_id:13},{$setOnInsert : {"name" : [null, "welt"]}, $set : {"name.0" : "wooorld"}}, { upsert: true });



 Comments   
Comment by Asya Kamsky [ 29/Jun/19 ]

SERVER-40381 implemented support for aggregation expressions to specify update for 4.2. 

You can see some examples here.

This can be done by setting the array to its new value using various aggregation array expressions based on conditional expression which can determine if the fields of interest already exist or not. I believe this can handle all of the examples/use cases that were mentioned in the comments.

Some of the examples given are no longer relevant in the world of WiredTiger but using a general example from one of the comments :

db.c.update({}, [
      {$set:{array:{$cond:{
            if: {$eq:[{$type:"$a"} ,  "missing"]}, // test if type of "$a" is "missing"
            then:{ } ,   // it's the upsert case
            else: { }    // it's the update case
      }}}}
]);

You can also have multiple stages, each one referencing/testing field coming from previous stage.

Comment by Asya Kamsky [ 03/Jun/19 ]

joniba this issue will be resolved in an upcoming version - I'll be updating the ticket with resolution later this month.

Comment by Jonathan Ben Ami [ 15/May/19 ]

Looks like this issue has been forgotten. As far as I can tell there are innumerable scenarios for this feature. If a document already exists, the consumer may want to set different data than if it does not yet exist. Any plan for this feature?

Comment by Maga Napanga [ 28/Apr/17 ]

@AsyaKamsky We thought to make it less expensive for the client so it can evaluate a boolean directly instead of comparing 2 date objects. Yours is of course valid, just every need is different.

Comment by Asya Kamsky [ 28/Apr/17 ]

But then won't the field just get set to false by the very next update? I'm not sure I see this use case being applicable here.

You can always set updated to timestamp and created to timestamp on insert only. Then you can always tell if the document has been updated since it was created by comparing those two fields.

Comment by Maga Napanga [ 28/Apr/17 ]

@Asya Kamsky That's not what I meant. I don't need to know that at the time the document is upserted, but later at any time when the document is requested by the client app. Thanks though.

Comment by Asya Kamsky [ 27/Apr/17 ]

maganap the update command will return a document which tells you whether the document was upserted.

WriteResult({ "nMatched" : 0, "nUpserted" : 1, "nModified" : 0, "_id" : 1 })

You should probably be testing that result to know if the document was updated or upserted.

Comment by Maga Napanga [ 25/Apr/17 ]

Another simple use case: I need a field to be true if the document was just inserted, and to be false if it was just updated.
If same field exists in both $set (as false) and $setOnInsert (as true), and the document is inserted, the field in $setOnInsert should take precedence even though it exists as well in the $set section.

Comment by Riccardo Cardin [ 06/Feb/17 ]

We have found this workaround http://stackoverflow.com/a/41953190/1173755 until the issue is resolved.

Comment by Kevin Rice [ 15/Apr/16 ]

+1 Initialize array and put first element in it is IMPOSSIBLE with out this.
(can only initialize a dict and put elements in that. Much more storage requirements).

Comment by Florian Winter [ 15/Jul/15 ]

+1 Much more simple use case where you run into this problem:

Atomically increment and get next value of a counter, creating and initializing the counter if it doesn't exist.

  db.findAndModify("mydb.counters", {_id: "myCounter"}, {$inc: {n: 1}, $setInsert: {n, 0}}, {upsert: true, new: true})

where "mydb.counters" is a collection of counters, "myCounter" is the name of one such counter, "n" is the counter's value. The query above increments the counter value by 1. If it doesn't exist, the counter is initialized with 0.

If this is allowed, then what should be the behavior? For the counter scenario, it would be best if only $setOnInsert (and not $inc) is applied if the counter doesn't exist. But I'm note sure if this is a good general rule.

Comment by Michel S. [ 03/Mar/15 ]

+1 - Same use case as most of the people here: Create a default value before modifying it using an upsert.

Comment by Jason Marmon [ 18/Dec/14 ]

+1 - we get data feeds that need to either update an existing document or create a new one; however, the updates only apply to certain portions of the document. i have to do two queries instead of one which is quite a performance hit when processing large data feeds

Comment by John Green [ 03/Dec/14 ]

Slightly different case, not involving $setOnInsert - I have a data model where client can add, update, delete
entries from an embedded array of documents. Currently, cannot use $set for the array field along with $push
and/or $pull. The only current solution is to $set the array as a whole.

Comment by Andre Spiegel [ 15/Sep/14 ]

+1 for a case where you want to pre-populate an array before inserting the first value

Comment by David Stevens [ 09/Sep/14 ]

+1 for use case creating multiple default values

Comment by Mike Lohmeier [ 20/Aug/14 ]

+1 running into this issue when trying to store sensor data in a time series collection.

Comment by Thomas Rueckstiess [ 15/Aug/14 ]

This issue is still present in 2.6:

MongoDB shell version: 2.6.3
connecting to: test
> db.foo.update({_id: 1}, {$setOnInsert: {a:0, b:0}, $inc: {a:1}}, {upsert: true})
WriteResult({
	"nMatched" : 0,
	"nUpserted" : 0,
	"nModified" : 0,
	"writeError" : {
		"code" : 16836,
		"errmsg" : "Cannot update 'a' and 'a' at the same time"
	}
})

It prevents a very common use case: Initialize counters when the document does not yet exist, or else increase counters.

Comment by Tim S [ 14/Aug/14 ]

+1 to fix this, simply ignore the $inc or have $setOnInsert override, can't be tough to do.

Comment by Patrick Shelby [ 09/Apr/14 ]

This is creating conflicts for my inserts as well. A workaround would be expensive in both the database and the inserting script.

Comment by William Cross [ 01/Apr/14 ]

I ran into this issue as well, with jacob.ribnik while making questions for an exam. I know he's run into it, separately.

Comment by Matthew Back [ 04/Dec/13 ]

This issue has become a problem for me.
I'm running mongod 2.4.8 using nodeJS mongodb driver version 1.3.20 and when I run

>db.mycollection.update({_id:{timestamp: new ISODate('2013-11-25T07:40:00Z'), action:'BK', protocol:'http'}},
{ $inc:

{'value.hits':1, 'value.download':10, 'value.upload':20}

,
$setOnInsert:{value:{hits:1, dowload:10, upload:20}}
},

{upsert:true, w:1}

)
in the mongo shell I also get the error "have conflicting mods in update" which I believe to be an incorrect error.
I need to find out if this fix is going to be released shortly or if I need to code a workaround.

Comment by Jon Rangel (Inactive) [ 20/Nov/13 ]

One use case for this is in pre-allocating documents for pre-aggregated reports, to avoid growing documents. The pre-allocation strategy advocated in the documentation is to allocate documents in batch up-front or probabilistically over some appropriate time interval. The drawback with this approach is that documents may be created that are not subsequently used, and are thus a waste of space.

For example, consider the following daily stats document with hourly buckets:

2.5.4|test> db.daily.update({day:6, app_id:123, event_id:456},{$setOnInsert:{counter1:0, counter2:0, hourly: [{counter1:0, counter2:0},{counter1:0, counter2:0}]}, $inc: {counter1:1, "hourly.0.counter1":1}},{upsert:true})
Update WriteResult({
	"ok" : 0,
	"code" : 16844,
	"errmsg" : "Cannot update 'counter1' and 'counter1' at the same time",
	"n" : 0
})
2.5.4|test>

In this example, a given event might not occur every day so it is undesirable to create documents for days when the event does not occur.

Generated at Thu Feb 08 03:23:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.