Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-13557

Incorrect negation of $elemMatch value in 2.6

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 2.6.1, 2.7.0
    • Affects Version/s: 2.6.0
    • Component/s: Querying
    • Labels:
      None
    • Environment:
    • ALL
    • Hide 
      > db.type_test.insert({"name": "Peter"})
> db.type_test.insert({"_id": "id1", "name": "Paula"})
> db.type_test.insert({"_id": "id2", "name": "Tim"})
> db.type_test.insert({"name": "Sara"})

> db.type_test.find()
{ "_id" : ObjectId("534812d3651139e8d4a9299d"), "name" : "Peter" }
{ "_id" : "id1", "name" : "Paula" }
{ "_id" : "id2", "name" : "Tim" }
{ "_id" : ObjectId("534812d5651139e8d4a9299e"), "name" : "Sara" }

      This should now show all 4 documents, but it does no longer (as it used to in 2.4):

      > db.type_test.find( { "_id" : { "$not" : { "$elemMatch" : { "$exists" : 1}}}} )
>

      Just checking that _id isn't all of a sudden handled as array

      > db.type_test.find( { "_id" : { "$elemMatch" : { "$exists" : 1}}} )
>

      It isn't.

      Note that for fields other than "_id" it still seems to work:

      > db.type_test.find( { "name" : { "$elemMatch" : { "$exists" : 1}}} )
> db.type_test.find( { "name" : { "$not" : { "$elemMatch" : { "$exists" : 1}}}} )
{ "_id" : ObjectId("534812d3651139e8d4a9299d"), "name" : "Peter" }
{ "_id" : "id1", "name" : "Paula" }
{ "_id" : "id2", "name" : "Tim" }
{ "_id" : ObjectId("534812d5651139e8d4a9299e"), "name" : "Sara" }
>
      Show
      > db.type_test.insert({ "name" : "Peter" }) > db.type_test.insert({ "_id" : "id1" , "name" : "Paula" }) > db.type_test.insert({ "_id" : "id2" , "name" : "Tim" }) > db.type_test.insert({ "name" : "Sara" }) > db.type_test.find() { "_id" : ObjectId( "534812d3651139e8d4a9299d" ), "name" : "Peter" } { "_id" : "id1" , "name" : "Paula" } { "_id" : "id2" , "name" : "Tim" } { "_id" : ObjectId( "534812d5651139e8d4a9299e" ), "name" : "Sara" } This should now show all 4 documents, but it does no longer (as it used to in 2.4): > db.type_test.find( { "_id" : { "$not" : { "$elemMatch" : { "$exists" : 1}}}} ) > Just checking that _id isn't all of a sudden handled as array > db.type_test.find( { "_id" : { "$elemMatch" : { "$exists" : 1}}} ) > It isn't. Note that for fields other than "_id" it still seems to work: > db.type_test.find( { "name" : { "$elemMatch" : { "$exists" : 1}}} ) > db.type_test.find( { "name" : { "$not" : { "$elemMatch" : { "$exists" : 1}}}} ) { "_id" : ObjectId( "534812d3651139e8d4a9299d" ), "name" : "Peter" } { "_id" : "id1" , "name" : "Paula" } { "_id" : "id2" , "name" : "Tim" } { "_id" : ObjectId( "534812d5651139e8d4a9299e" ), "name" : "Sara" } >

      Issue Status as of April 15, 2014

      ISSUE SUMMARY
      By definition, a negated $elemMatch value should return all results for which the field is not an array. This behavior is incorrect in 2.6.0 if the field is indexed.

      USER IMPACT
      Users may be getting unexpected results with indexed negated $elemMatch queries. Additionally, in SERVER-1475, a work-around to detect whether or not a field is an array was suggested using $elemMatch. This work-around no longer works in 2.6.0, due to the above bug.

      WORKAROUNDS
      None

      RESOLUTION
      Prevent negated $elemMatch values to use an index scan.

      AFFECTED VERSIONS
      Version 2.6.0 is affected by this bug.

      PATCHES
      The patch is included in the 2.6.1 production release.

      Original description

      A negated $elemMatch value should return all results for which the field is not an array. This behavior is incorrect in 2.6.0 if the field is indexed:

      > t.drop()
> t.ensureIndex({a: 1})
> t.save({a: 2})
> t.save({a: [1, 2, 3]})
> t.find()
{ "_id" : ObjectId("534bfa1a99d4c7138703741f"), "a" : 2 }
{ "_id" : ObjectId("534bfa2099d4c71387037420"), "a" : [ 1, 2, 3 ] }
> t.find({a: {$elemMatch: {$gt: 1}}})
{ "_id" : ObjectId("534bfa2099d4c71387037420"), "a" : [ 1, 2, 3 ] } // this is the expected result
> t.find({a: {$not: {$elemMatch: {$gt: 1}}}}) // this returns no results, but should return doc {a: 2}
> t.dropIndexes()
> t.find({a: {$not: {$elemMatch: {$gt: 1}}}})
{ "_id" : ObjectId("534bfa1a99d4c7138703741f"), "a" : 2 } // correct result without the index

      Original description:

      In SERVER-1475, a nice $type work-around to detect whether or not a field is an array was suggested using $elemMatch.
      I much prefer that approach over the now suggested route via $where, which I suspect to be very, very slow.
      Bizzarrely, as of 2.6.0 $elemMatch seems to be broken in combination with the "_id" field, other fields do not appear to be affected.

            Assignee:
            david.storch@mongodb.com David Storch
            Reporter:
            tomzahn Thomas Zahn
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: