Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-13557

Incorrect negation of $elemMatch value in 2.6

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.6.1, 2.7.0
    • Component/s: Querying
    • Labels:
      None
    • Environment:
    • Operating System:
      ALL
    • Backport Completed:
    • Steps To Reproduce:
      Hide

      > db.type_test.insert({"name": "Peter"})
      > db.type_test.insert({"_id": "id1", "name": "Paula"})
      > db.type_test.insert({"_id": "id2", "name": "Tim"})
      > db.type_test.insert({"name": "Sara"})
       
      > db.type_test.find()
      { "_id" : ObjectId("534812d3651139e8d4a9299d"), "name" : "Peter" }
      { "_id" : "id1", "name" : "Paula" }
      { "_id" : "id2", "name" : "Tim" }
      { "_id" : ObjectId("534812d5651139e8d4a9299e"), "name" : "Sara" } 

      This should now show all 4 documents, but it does no longer (as it used to in 2.4):

      > db.type_test.find( { "_id" : { "$not" : { "$elemMatch" : { "$exists" : 1}}}} )
      >

      Just checking that _id isn't all of a sudden handled as array

      > db.type_test.find( { "_id" : { "$elemMatch" : { "$exists" : 1}}} )
      >

      It isn't.

      Note that for fields other than "_id" it still seems to work:

      > db.type_test.find( { "name" : { "$elemMatch" : { "$exists" : 1}}} )
      > db.type_test.find( { "name" : { "$not" : { "$elemMatch" : { "$exists" : 1}}}} )
      { "_id" : ObjectId("534812d3651139e8d4a9299d"), "name" : "Peter" }
      { "_id" : "id1", "name" : "Paula" }
      { "_id" : "id2", "name" : "Tim" }
      { "_id" : ObjectId("534812d5651139e8d4a9299e"), "name" : "Sara" }
      > 

      Show
      > db.type_test.insert({"name": "Peter"}) > db.type_test.insert({"_id": "id1", "name": "Paula"}) > db.type_test.insert({"_id": "id2", "name": "Tim"}) > db.type_test.insert({"name": "Sara"})   > db.type_test.find() { "_id" : ObjectId("534812d3651139e8d4a9299d"), "name" : "Peter" } { "_id" : "id1", "name" : "Paula" } { "_id" : "id2", "name" : "Tim" } { "_id" : ObjectId("534812d5651139e8d4a9299e"), "name" : "Sara" } This should now show all 4 documents, but it does no longer (as it used to in 2.4): > db.type_test.find( { "_id" : { "$not" : { "$elemMatch" : { "$exists" : 1}}}} ) > Just checking that _id isn't all of a sudden handled as array > db.type_test.find( { "_id" : { "$elemMatch" : { "$exists" : 1}}} ) > It isn't. Note that for fields other than "_id" it still seems to work: > db.type_test.find( { "name" : { "$elemMatch" : { "$exists" : 1}}} ) > db.type_test.find( { "name" : { "$not" : { "$elemMatch" : { "$exists" : 1}}}} ) { "_id" : ObjectId("534812d3651139e8d4a9299d"), "name" : "Peter" } { "_id" : "id1", "name" : "Paula" } { "_id" : "id2", "name" : "Tim" } { "_id" : ObjectId("534812d5651139e8d4a9299e"), "name" : "Sara" } >

      Description

      Issue Status as of April 15, 2014

      ISSUE SUMMARY
      By definition, a negated $elemMatch value should return all results for which the field is not an array. This behavior is incorrect in 2.6.0 if the field is indexed.

      USER IMPACT
      Users may be getting unexpected results with indexed negated $elemMatch queries. Additionally, in SERVER-1475, a work-around to detect whether or not a field is an array was suggested using $elemMatch. This work-around no longer works in 2.6.0, due to the above bug.

      WORKAROUNDS
      None

      RESOLUTION
      Prevent negated $elemMatch values to use an index scan.

      AFFECTED VERSIONS
      Version 2.6.0 is affected by this bug.

      PATCHES
      The patch is included in the 2.6.1 production release.

      Original description

      A negated $elemMatch value should return all results for which the field is not an array. This behavior is incorrect in 2.6.0 if the field is indexed:

      > t.drop()
      > t.ensureIndex({a: 1})
      > t.save({a: 2})
      > t.save({a: [1, 2, 3]})
      > t.find()
      { "_id" : ObjectId("534bfa1a99d4c7138703741f"), "a" : 2 }
      { "_id" : ObjectId("534bfa2099d4c71387037420"), "a" : [ 1, 2, 3 ] }
      > t.find({a: {$elemMatch: {$gt: 1}}})
      { "_id" : ObjectId("534bfa2099d4c71387037420"), "a" : [ 1, 2, 3 ] } // this is the expected result
      > t.find({a: {$not: {$elemMatch: {$gt: 1}}}}) // this returns no results, but should return doc {a: 2}
      > t.dropIndexes()
      > t.find({a: {$not: {$elemMatch: {$gt: 1}}}})
      { "_id" : ObjectId("534bfa1a99d4c7138703741f"), "a" : 2 } // correct result without the index

      Original description:

      In SERVER-1475, a nice $type work-around to detect whether or not a field is an array was suggested using $elemMatch.
      I much prefer that approach over the now suggested route via $where, which I suspect to be very, very slow.
      Bizzarrely, as of 2.6.0 $elemMatch seems to be broken in combination with the "_id" field, other fields do not appear to be affected.

        Attachments

          Activity

            People

            • Votes:
              2 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: