[SERVER-34973] Upsert querying an array, then pushing to this array with simple values Created: 14/May/18  Updated: 27/Oct/23  Resolved: 21/May/18

Status: Closed
Project: Core Server
Component/s: Write Ops
Affects Version/s: 3.6.4
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Arthur Darcet Assignee: Asya Kamsky
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-6566 Support conditional updates: $updates Closed
Related
is related to SERVER-20203 upsert querying an array field and $p... Closed
Operating System: ALL
Steps To Reproduce:

db.empty.update({"list": "value"}, {"$push": {list: "value", {upsert: 1});
The field 'list' must be an array but is of type string in document {no id}

db.empty.update({"list": {"$elemMatch": "value", {"$push": {list: "value"}}, {upsert: 1});}}
$elemMatch needs an Object

db.empty.update({"list": {"$in": ["value"], {"$push": {list: "value"}}, {upsert: 1});}}
The field 'list' must be an array but is of type string in document {no id}

db.empty.update({"list": {$in: ["value", "___wrong"], {"$push": {list: "value"}}, {upsert: 1});}}
// success

Participants:

 Description   

As stated in the docs:

The update creates the new document with [...] the fields and values of both the <query> and <update> parameters if the <update> parameter contains update operator expressions. The update creates a base document from the equality clauses in the <query> parameter, and then applies the update expressions from the <update> parameter.

 

So of course, this fails:

db.empty.update({"list": "value"}, {"$push": {list: "value", {upsert: 1});

with

The field 'list' must be an array but is of type string in document {no id}

The workaround, suggested in https://jira.mongodb.org/browse/SERVER-20203 is to add an $elemMatch to the query, so that the upserted document is not "pre-filled" with a scalar value for the list key.

But this does not work here: $elemMatch refuses to match single values, so:

db.empty.update({"list": {"$elemMatch": "value", {"$push": {list: "value"}}, {upsert: 1});}}

yields

$elemMatch needs an Object

  

If I remember correctly, a working workaround used to be:

db.empty.update({"list": {"$in": ["value"], {"$push": {list: "value"}}, {upsert: 1});}}

in 3.2 or 3.4 ; but $in with single value arrays are considered equality matches in 3.6 (i'm guessing here), and this last option fails with

The field 'list' must be an array but is of type string in document {no id}

again.

The only way I found to do this is

db.empty.update({"list": {$in: ["value", "___wrong"], {"$push": {list: "value"}}, {upsert: 1});}}

but this seems very inefficient and I would like to have a cleaner option



 Comments   
Comment by Arthur Darcet [ 22/May/18 ]

Yes thank you, the $eq solution works

Comment by Asya Kamsky [ 21/May/18 ]

I'm closing this issue as I believe your use case will work with syntax I showed.

If that's not the case please reopen (and then likely it will be a duplicate of SERVER-6566.

Comment by Asya Kamsky [ 21/May/18 ]

I think maybe you misunderstood the error message here:

db.empty.update({identifiers: {$elemMatch: "id-new"}}, {$push: {identifiers: "id-new"}, $set: …}, {upsert: 1});
$elemMatch needs an Object 

That's saying that you are not giving the right syntax to $elemMatch, which expects the query to be an object.

You can use this syntax to get $elemMatch to work:

db.empty.update({identifiers: {$elemMatch: {$eq:"id-new"}}}, {$push: {identifiers: "id-new"}}, {upsert: 1});
WriteResult({WriteResult({ "nMatched" : 0, "nUpserted" : 1, "nModified" : 0, "_id" : ObjectId("5b030bea07d2f50c20b247a7")})

Comment by Arthur Darcet [ 21/May/18 ]

Hello @asya,

Thank you for your response. Please let me clarify my request a bit, I probably did not include enough context:

My use-case is quite simple: I have objects identified by several strings, and I would like to set an attribute on a given identifier - creating it first if it does not yet exist.

The model is just:

{identifiers: ["id1, "id2", …], attribute: 42}

On all the documents in the collection, `identifiers` is an array. And I need to ensure there is no duplicates, ie any identifier is only present in one document at most.


Trying to set atomically `attribute: 100` on an identifier that is not yet in the DB means doing an upsert, but this upsert does not work because if I match {identifiers: "id3"}, then the upsert defaults the value of `identifiers` to a string and not an array, which would not work.

>>> db.empty.update({ identifiers: "id-new" }, { $push: { identifiers: "id-new" }, $set: { attribute: 100 } }, {upsert: 1});
The field 'list' must be an array but is of type string in document {no id}


The workaround given in SERVER-20203 is to add a $elemMatch in the query, so that the upsert does not use it to set default values, which then allows a {$setOnInsert: {identifiers: ["id3"]}} which would work.

But of course $elemMatch does not work with scalar values, so this workaround is not available here.

db.empty.update({identifiers: {$elemMatch: "id-new"}}, {$push: {identifiers: "id-new"}, $set: …}, {upsert: 1});
$elemMatch needs an Object


Another workaround that did work until 3.6 is too wrap the query in an $in, using {identifiers: {$in: ["id3"]}} ; but since the 3.6, this is single-value $in is optimised away, and then the upsert fill the new document with a string value for identifiers

db.empty.update({identifiers: {$in: ["id-new"], {$push: {identifiers: "id-new"}, $set: …}, {upsert: 1});
The field 'list' must be an array but is of type string in document {no id}

Which leaves us only one option: adding a second, completely wrong value in the query array for $in. This is inefficient, and ugly…

db.empty.update({identifiers: {$in: ["id-new", "___wrong"]}}, {$push: {identifiers: "id-new"}, $set: …}, {upsert: 1});


I think my use-case would indeed be covered by SERVER-6566, by doing something like

db.empty.update(
    {identifiers: "id-new"},
    [{
        $when: {identifiers: null},
        $do: { $set: { identifiers: ["id3"] } },
    }]
);

but my use-case seems simple enough, and i think it shouldn't need to use a feature so complex… (+ this won't be available in the stable release for some time…)

Comment by Asya Kamsky [ 18/May/18 ]

arthur,

This is working as expected. It is never going to be correct to try to $push anything to a field which is not of type array.
You say about the suggested $elemMatch workaround:

> But this does not work here: $elemMatch refuses to match single values, so:
> db.empty.update({"list": {"$elemMatch": "value"}}, {"$push": {list: "value"}},{upsert: 1});

It cannot work, because when list is a single value (i.e. not an array) you cannot $push to it.

Your other workaround with additional "never present" value in $in array works for an upsert, but it will give the same error if "value" already exists as a single value (the case you are worried about missing):

db.empty.insert({list:"value"})
WriteResult({ "nInserted" : 1 })
db.empty.update({"list": {$in: ["value", "___wrong"]}}, {"$push": {list: "value"}}, {upsert: 1})
WriteResult({
	"nMatched" : 0,
	"nUpserted" : 0,
	"nModified" : 0,
	"writeError" : {
		"code" : 2,
		"errmsg" : "The field 'list' must be an array but is of type string in document {_id: ObjectId('5aff1ab7e7342a0dcecdf627')}"
	}
})

I suspect that you want to treat single value list as an array with a single element, but currently there is no support for such behavior (converting the type of a field during an update, or treating a string value as if it were an array with a single value that you can $push to).

If I'm correct and your use case is for update to perform a different transformation when matched value is an array vs. when it's another type then that would be similar to conditional updates, tracked in SERVER-6566.   Does that seem like it would address your use case?

 

Generated at Thu Feb 08 04:38:25 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.