Core Server
  1. Core Server
  2. SERVER-8172

a query field with a $not operator should not be used to populate an upsert document

    Details

    • Backport:
      Done
    • Backward Breaking:
      No
    • Operating System:
      ALL
    • Bug Type:
      Logical
    • # Replies:
      5
    • Last comment by Customer:
      false
    • Steps To Reproduce:
      Hide

      db.test.update({_id: 1, r: {$not: {$elemMatch: {"a": 1}}}}, {$push: {r:

      {"a": 1, "e": 0}

      }},

      {upsert: true}

      );

      Show
      db.test.update({_id: 1, r: {$not: {$elemMatch: {"a": 1}}}}, {$push: {r: {"a": 1, "e": 0} }}, {upsert: true} );
    • Documentation changes needed?:
      Yes

      Description

      In 2.0.8, createNewFromQuery performs the following check to see if a
      query field represents a value that should be incorporated in the
      upsert document, or an operator:

      if ( e.type() == Object && e.embeddedObject().firstElementFieldName()[0] == '$' ) {
          // this means this is a $gt type filter, so don't make part of the new object continue;
      }
      

      In 2.2.2, that check became the following:

      if ( e.type() == Object && e.embeddedObject().firstElementFieldName()[0] == '$' ) {
          // we have something like { x : { $gt : 5 } }
          // this can be a query piece
          // or can be a dbref or something
       
          int op = e.embeddedObject().firstElement().getGtLtOp( -1 );
          if ( op >= 0 ) {
              // this means this is a $gt type filter, so don't make part of the new object continue;
          }
      }
      

      Now we only consider the object field to represent an operator if
      getGtLtOp() can interpret the field name as an operator. However,
      this is not a complete check because getGtLtOp() only understands
      simple field-value operators. It does not parse $not (or $and, etc.).
      This means that the {$not: {$elemMatch:

      {"a": 1}

      }} portion of the
      query is interpreted by the upsert implementation a value, not an
      operator. The upserted document starts with a value of the 'r' field
      of {$not: {$elemMatch:

      {"a": 1}

      }}. $push cannot operate on this value
      because it is not an array, which causes the error the user reported.

      Aaron

      ------------------------------------

      In 2.2.x $elemMatch breaks $push when used inside of $not and with an upsert. This is useful is you want to push only when an existing property doesn't exist inside of an array.

      An update in 2.0 such as:
      db.test.update({_id: 1, r: {$not: {$elemMatch: {"a": 1}}}}, {$push: {r:

      {"a": 1, "e": 0}

      }},

      {upsert: true}

      );
      or even:
      db.test.update({_id: 1, r: {$not: {$elemMatch: {$or: [

      {"a": 1}

      ,

      {"e": 1}

      ]}}}}, {$push: {r:

      {"a": 1, "e": 0}

      }},

      {upsert: true}

      );
      Would successfully create a new document:
      { "_id" : 1, "r" : [

      { "a" : 1, "e" : 0 }

      ] }

      If you called either of those updates again, it would expectantly throw a 11000 error:
      E11000 duplicate key error index: test.test.$id dup key: { : "hello" }
      Which meant that there already was a value inside "r" with either

      {"a": 1}

      or

      {"e": 1}

      .

      Now in 2.2 either of those updates fails with error:
      Cannot apply $push/$pushAll modifier to non-array

      There was a proposed workaround (using $nin): http://stackoverflow.com/questions/12391334/mongodb-upsert-with-push-and-not
      But that doesn't actually work. The $nin fails to actually look inside the array, which is expected as it wouldn't seem that it would anyways.

        Activity

        Hide
        James Hartig
        added a comment -

        This is slated for 2.4.x. Can we get this fixed in 2.2.x?

        Show
        James Hartig
        added a comment - This is slated for 2.4.x. Can we get this fixed in 2.2.x?
        Hide
        auto
        added a comment -

        Author:

        {u'date': u'2013-01-16T21:47:34Z', u'email': u'aaron@10gen.com', u'name': u'aaron'}

        Message: SERVER-8172 Exclude $not query operator fields from new upsert document.
        Branch: master
        https://github.com/mongodb/mongo/commit/d30f8de98c08a1fdc599dc717758dc0edf719083

        Show
        auto
        added a comment - Author: {u'date': u'2013-01-16T21:47:34Z', u'email': u'aaron@10gen.com', u'name': u'aaron'} Message: SERVER-8172 Exclude $not query operator fields from new upsert document. Branch: master https://github.com/mongodb/mongo/commit/d30f8de98c08a1fdc599dc717758dc0edf719083
        Hide
        James Hartig
        added a comment -

        Instead of another if checking for just $not, should there be another method like getGtLtOp that checks for $gt/$lt and other operators too? Is $and/$or going to be broken?

        Show
        James Hartig
        added a comment - Instead of another if checking for just $not, should there be another method like getGtLtOp that checks for $gt/$lt and other operators too? Is $and/$or going to be broken?
        Hide
        Aaron Staple (Inactive)
        added a comment -

        $not is the only additional operator valid in this context. $and is not a field-value operator and cannot be correctly supplied here in a query.

        We can revisit the implementation choice after SERVER-7391.

        Show
        Aaron Staple (Inactive)
        added a comment - $not is the only additional operator valid in this context. $and is not a field-value operator and cannot be correctly supplied here in a query. We can revisit the implementation choice after SERVER-7391 .
        Hide
        auto
        added a comment -

        Author:

        {u'date': u'2013-01-16T21:47:34Z', u'name': u'aaron', u'email': u'aaron@10gen.com'}

        Message: SERVER-8172 Exclude $not query operator fields from new upsert document.

        Conflicts:

        src/mongo/dbtests/updatetests.cpp
        Branch: v2.2
        https://github.com/mongodb/mongo/commit/8ea5dd424949d66d0859a45e9b5bc0889fb0ca30

        Show
        auto
        added a comment - Author: {u'date': u'2013-01-16T21:47:34Z', u'name': u'aaron', u'email': u'aaron@10gen.com'} Message: SERVER-8172 Exclude $not query operator fields from new upsert document. Conflicts: src/mongo/dbtests/updatetests.cpp Branch: v2.2 https://github.com/mongodb/mongo/commit/8ea5dd424949d66d0859a45e9b5bc0889fb0ca30

          People

          • Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Days since reply:
              1 year, 4 weeks, 2 days ago
              Date of 1st Reply: