[SERVER-13557] Incorrect negation of $elemMatch value in 2.6 Created: 11/Apr/14  Updated: 11/Jul/16  Resolved: 14/Apr/14

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: 2.6.0
Fix Version/s: 2.6.1, 2.7.0

Type: Bug Priority: Major - P3
Reporter: Thomas Zahn Assignee: David Storch
Resolution: Done Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

> db.serverBuildInfo()
{
"version" : "2.6.0",
"gitVersion" : "1c1c76aeca21c5983dc178920f5052c298db616c",
"OpenSSLVersion" : "",
"sysInfo" : "Darwin mci-osx108-8.build.10gen.cc 12.5.0 Darwin Kernel Version 12.5.0: Sun Sep 29 13:33:47 PDT 2013; root:xnu-2050.48.12~1/RELEASE_X86_64 x86_64 BOOST_LIB_VERSION=1_49",
"loaderFlags" : "-fPIC -pthread -Wl,-bind_at_load -mmacosx-version-min=10.6",
"compilerFlags" : "-Wnon-virtual-dtor -Woverloaded-virtual -fPIC -fno-strict-aliasing -ggdb -pthread -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -Werror -pipe -O3 -Wno-unused-function -Wno-deprecated-declarations -mmacosx-version-min=10.6",
"allocator" : "system",
"versionArray" : [
2,
6,
0,
0
],
"javascriptEngine" : "V8",
"bits" : 64,
"debug" : false,
"maxBsonObjectSize" : 16777216,
"ok" : 1
}


Issue Links:
Related
Operating System: ALL
Backport Completed:
Steps To Reproduce:

> db.type_test.insert({"name": "Peter"})
> db.type_test.insert({"_id": "id1", "name": "Paula"})
> db.type_test.insert({"_id": "id2", "name": "Tim"})
> db.type_test.insert({"name": "Sara"})
 
> db.type_test.find()
{ "_id" : ObjectId("534812d3651139e8d4a9299d"), "name" : "Peter" }
{ "_id" : "id1", "name" : "Paula" }
{ "_id" : "id2", "name" : "Tim" }
{ "_id" : ObjectId("534812d5651139e8d4a9299e"), "name" : "Sara" } 

This should now show all 4 documents, but it does no longer (as it used to in 2.4):

> db.type_test.find( { "_id" : { "$not" : { "$elemMatch" : { "$exists" : 1}}}} )
>

Just checking that _id isn't all of a sudden handled as array

> db.type_test.find( { "_id" : { "$elemMatch" : { "$exists" : 1}}} )
>

It isn't.

Note that for fields other than "_id" it still seems to work:

> db.type_test.find( { "name" : { "$elemMatch" : { "$exists" : 1}}} )
> db.type_test.find( { "name" : { "$not" : { "$elemMatch" : { "$exists" : 1}}}} )
{ "_id" : ObjectId("534812d3651139e8d4a9299d"), "name" : "Peter" }
{ "_id" : "id1", "name" : "Paula" }
{ "_id" : "id2", "name" : "Tim" }
{ "_id" : ObjectId("534812d5651139e8d4a9299e"), "name" : "Sara" }
> 

Participants:

 Description   
Issue Status as of April 15, 2014

ISSUE SUMMARY
By definition, a negated $elemMatch value should return all results for which the field is not an array. This behavior is incorrect in 2.6.0 if the field is indexed.

USER IMPACT
Users may be getting unexpected results with indexed negated $elemMatch queries. Additionally, in SERVER-1475, a work-around to detect whether or not a field is an array was suggested using $elemMatch. This work-around no longer works in 2.6.0, due to the above bug.

WORKAROUNDS
None

RESOLUTION
Prevent negated $elemMatch values to use an index scan.

AFFECTED VERSIONS
Version 2.6.0 is affected by this bug.

PATCHES
The patch is included in the 2.6.1 production release.

Original description

A negated $elemMatch value should return all results for which the field is not an array. This behavior is incorrect in 2.6.0 if the field is indexed:

> t.drop()
> t.ensureIndex({a: 1})
> t.save({a: 2})
> t.save({a: [1, 2, 3]})
> t.find()
{ "_id" : ObjectId("534bfa1a99d4c7138703741f"), "a" : 2 }
{ "_id" : ObjectId("534bfa2099d4c71387037420"), "a" : [ 1, 2, 3 ] }
> t.find({a: {$elemMatch: {$gt: 1}}})
{ "_id" : ObjectId("534bfa2099d4c71387037420"), "a" : [ 1, 2, 3 ] } // this is the expected result
> t.find({a: {$not: {$elemMatch: {$gt: 1}}}}) // this returns no results, but should return doc {a: 2}
> t.dropIndexes()
> t.find({a: {$not: {$elemMatch: {$gt: 1}}}})
{ "_id" : ObjectId("534bfa1a99d4c7138703741f"), "a" : 2 } // correct result without the index

Original description:

In SERVER-1475, a nice $type work-around to detect whether or not a field is an array was suggested using $elemMatch.
I much prefer that approach over the now suggested route via $where, which I suspect to be very, very slow.
Bizzarrely, as of 2.6.0 $elemMatch seems to be broken in combination with the "_id" field, other fields do not appear to be affected.



 Comments   
Comment by Githook User [ 14/Apr/14 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-13557 disallow negated elemMatch value from using an index scan

(cherry picked from commit 45cf39c95d0fdfcb15b483e2d50b392a95735043)
Branch: v2.6
https://github.com/mongodb/mongo/commit/ccb02e83df89baebd889791c33dd72122a0f43b4

Comment by Githook User [ 14/Apr/14 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-13557 disallow negated elemMatch value from using an index scan
Branch: master
https://github.com/mongodb/mongo/commit/45cf39c95d0fdfcb15b483e2d50b392a95735043

Generated at Thu Feb 08 03:32:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.