[SERVER-8172] a query field with a $not operator should not be used to populate an upsert document Created: 15/Jan/13  Updated: 28/Oct/15  Resolved: 18/Jan/13

Status: Closed
Project: Core Server
Component/s: Querying, Write Ops
Affects Version/s: 2.2.2
Fix Version/s: 2.2.4, 2.4.0-rc0

Type: Bug Priority: Major - P3
Reporter: James Hartig Assignee: Aaron Staple
Resolution: Done Votes: 1
Labels: elemmatch, update, upsert
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

MongoDB v2.2.2 or MongoDB v2.0.6
Running CentOS


Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

db.test.update({_id: 1, r: {$not: {$elemMatch: {"a": 1}}}}, {$push: {r:

{"a": 1, "e": 0}

}},

{upsert: true}

);

Participants:

 Description   

In 2.0.8, createNewFromQuery performs the following check to see if a
query field represents a value that should be incorporated in the
upsert document, or an operator:

if ( e.type() == Object && e.embeddedObject().firstElementFieldName()[0] == '$' ) {
    // this means this is a $gt type filter, so don't make part of the new object continue;
}

In 2.2.2, that check became the following:

if ( e.type() == Object && e.embeddedObject().firstElementFieldName()[0] == '$' ) {
    // we have something like { x : { $gt : 5 } }
    // this can be a query piece
    // or can be a dbref or something
 
    int op = e.embeddedObject().firstElement().getGtLtOp( -1 );
    if ( op >= 0 ) {
        // this means this is a $gt type filter, so don't make part of the new object continue;
    }
}

Now we only consider the object field to represent an operator if
getGtLtOp() can interpret the field name as an operator. However,
this is not a complete check because getGtLtOp() only understands
simple field-value operators. It does not parse $not (or $and, etc.).
This means that the {$not: {$elemMatch:

{"a": 1}

}} portion of the
query is interpreted by the upsert implementation a value, not an
operator. The upserted document starts with a value of the 'r' field
of {$not: {$elemMatch:

{"a": 1}

}}. $push cannot operate on this value
because it is not an array, which causes the error the user reported.

Aaron

------------------------------------

In 2.2.x $elemMatch breaks $push when used inside of $not and with an upsert. This is useful is you want to push only when an existing property doesn't exist inside of an array.

An update in 2.0 such as:
db.test.update({_id: 1, r: {$not: {$elemMatch: {"a": 1}}}}, {$push: {r:

{"a": 1, "e": 0}

}},

{upsert: true}

);
or even:
db.test.update({_id: 1, r: {$not: {$elemMatch: {$or: [

{"a": 1}

,

{"e": 1}

]}}}}, {$push: {r:

{"a": 1, "e": 0}

}},

{upsert: true}

);
Would successfully create a new document:
{ "_id" : 1, "r" : [

{ "a" : 1, "e" : 0 }

] }

If you called either of those updates again, it would expectantly throw a 11000 error:
E11000 duplicate key error index: test.test.$id dup key: { : "hello" }
Which meant that there already was a value inside "r" with either

{"a": 1}

or

{"e": 1}

.

Now in 2.2 either of those updates fails with error:
Cannot apply $push/$pushAll modifier to non-array

There was a proposed workaround (using $nin): http://stackoverflow.com/questions/12391334/mongodb-upsert-with-push-and-not
But that doesn't actually work. The $nin fails to actually look inside the array, which is expected as it wouldn't seem that it would anyways.



 Comments   
Comment by auto [ 21/Mar/13 ]

Author:

{u'date': u'2013-01-16T21:47:34Z', u'name': u'aaron', u'email': u'aaron@10gen.com'}

Message: SERVER-8172 Exclude $not query operator fields from new upsert document.

Conflicts:

src/mongo/dbtests/updatetests.cpp
Branch: v2.2
https://github.com/mongodb/mongo/commit/8ea5dd424949d66d0859a45e9b5bc0889fb0ca30

Comment by Aaron Staple [ 16/Jan/13 ]

$not is the only additional operator valid in this context. $and is not a field-value operator and cannot be correctly supplied here in a query.

We can revisit the implementation choice after SERVER-7391.

Comment by James Hartig [ 16/Jan/13 ]

Instead of another if checking for just $not, should there be another method like getGtLtOp that checks for $gt/$lt and other operators too? Is $and/$or going to be broken?

Comment by auto [ 16/Jan/13 ]

Author:

{u'date': u'2013-01-16T21:47:34Z', u'email': u'aaron@10gen.com', u'name': u'aaron'}

Message: SERVER-8172 Exclude $not query operator fields from new upsert document.
Branch: master
https://github.com/mongodb/mongo/commit/d30f8de98c08a1fdc599dc717758dc0edf719083

Comment by James Hartig [ 16/Jan/13 ]

This is slated for 2.4.x. Can we get this fixed in 2.2.x?

Generated at Thu Feb 08 03:16:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.