[SERVER-2239] indexes causing non-distinct results in $or query Created: 16/Dec/10  Updated: 04/Feb/15  Resolved: 14/Feb/11

Status: Closed
Project: Core Server
Component/s: Index Maintenance, Querying, Usability
Affects Version/s: 1.6.3, 1.6.4, 1.6.5
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Amos King Assignee: Aaron Staple
Resolution: Duplicate Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

OSX, CentOS, Linux, 32 and 64 bit


Issue Links:
Depends
depends on SERVER-2245 $or deduping should use IndexSpec get... Closed
Operating System: ALL
Participants:

 Description   

I'm not sure how to describe this. So I made a gist.

https://gist.github.com/743921

If I run the query in question with only one object in the collection I get back two copies of the object.

Indexes
======
> db.system.indexes.find()
{ "name" : "id", "ns" : "t1d_development.users", "key" :

{ "_id" : 1 }

}
{ "name" : "email_1_encrypted_password_1", "ns" : "t1d_development.users", "key" :

{ "email" : 1, "encrypted_password" : 1 }

, "unique" : true }
{ "name" : "memberships_1", "ns" : "t1d_development.users", "key" :

{ "memberships" : 1 }

, "unique" : false }
{ "name" : "acl_fields.public_1", "ns" : "t1d_development.users", "key" :

{ "acl_fields.public" : 1 }

, "unique" : false }
{ "name" : "acl_fields.value_1", "ns" : "t1d_development.users", "key" :

{ "acl_fields.value" : 1 }

, "unique" : false }
{ "name" : "acl_fields.circles_1", "ns" : "t1d_development.users", "key" :

{ "acl_fields.circles" : 1 }

, "unique" : false }

Object
=====
db.users.findOne()
{
"_id" : ObjectId("4d0a54246a931656bc000006"),
"acl_fields" : [

{ "circles" : [ ], "name" : "first_name", "public" : false, "value" : "Amos", "_id" : ObjectId("4d0a54246a931656bc000005"), "_type" : "AccessControlledFields::AclFirstName" }

,

{ "circles" : [ ], "name" : "email", "public" : false, "value" : "amos@amos.com", "_id" : ObjectId("4d0a54246a931656bc000007"), "_type" : "AccessControlledFields::AclEmail" }

,

{ "circles" : [ ], "name" : "nickname", "public" : true, "value" : "aking", "_id" : ObjectId("4d0a54686a931656bc000009"), "_type" : "AccessControlledFields::AclNickname" }

,

{ "circles" : [ ], "name" : "race_ethnicity", "public" : false, "value" : [ ], "_id" : ObjectId("4d0a62316a931656bc00000a"), "_type" : "AccessControlledFields::AclRaceEthnicity" }

,

{ "circles" : [ ], "name" : "middle_name", "public" : false, "value" : "", "_id" : ObjectId("4d0a62316a931656bc00000b"), "_type" : "AccessControlledFields::AclMiddleName" }

,

{ "circles" : [ ], "name" : "last_name", "public" : false, "value" : "", "_id" : ObjectId("4d0a62316a931656bc00000c"), "_type" : "AccessControlledFields::AclLastName" }

,

{ "circles" : [ ], "name" : "gender", "public" : false, "value" : "Male", "_id" : ObjectId("4d0a62316a931656bc00000d"), "_type" : "AccessControlledFields::AclGender" }

,

{ "circles" : [ ], "name" : "zip_code", "public" : false, "value" : "", "_id" : ObjectId("4d0a62316a931656bc00000e"), "_type" : "AccessControlledFields::AclZipCode" }

,

{ "circles" : [ ], "name" : "date_of_birth", "public" : false, "value" : null, "_id" : ObjectId("4d0a62316a931656bc00000f"), "_type" : "AccessControlledFields::AclDateOfBirth" }

],
"confirmation_sent_at" : "Thu Dec 16 2010 12:02:12 GMT-0600 (CST)",
"confirmation_token" : null,
"confirmed_at" : "Thu Dec 16 2010 12:03:20 GMT-0600 (CST)",
"connections_circle_of_trust" :

{ "name" : "Connections", "email" : false, "date_of_birth" : false, "race_ethnicity" : false, "gender" : false, "first_name" : false, "middle_name" : false, "last_name" : false, "zip_code" : false, "_id" : ObjectId("4d0a54246a931656bc000008") }

,
"created_at" : "Thu Dec 16 2010 12:02:12 GMT-0600 (CST)",
"current_sign_in_at" : "Thu Dec 16 2010 12:03:20 GMT-0600 (CST)",
"current_sign_in_ip" : "127.0.0.1",
"email" : "amos@amos.com",
"encrypted_password" : "$2a$10$/dgVQhHdIYjSJa6yz6RWbezcuVjzwT1rj70PpybwjzN56i8X8.aBu",
"first_name" : "Amos",
"gender" : "Male",
"last_name" : "",
"last_sign_in_at" : "Thu Dec 16 2010 12:03:20 GMT-0600 (CST)",
"last_sign_in_ip" : "127.0.0.1",
"memberships" : [ ],
"middle_name" : "",
"nickname" : "aking",
"password_salt" : "$2a$10$/dgVQhHdIYjSJa6yz6RWbe",
"race_ethnicity" : [ ],
"sign_in_count" : 1,
"terms_acceptance" : "1",
"updated_at" : "Thu Dec 16 2010 13:02:09 GMT-0600 (CST)",
"zip_code" : ""
}

Query
=====
db.users.find({$or:[
{acl_fields:{$elemMatch:{name:{$ne:"gender"}, circles:{$in:[]}, value : /aking/i}}},
{acl_fields:{$elemMatch:{name:"gender", circles:{$in:[]}, value : /^aking$/i}}},
{acl_fields:{$elemMatch:

{name:"gender", public:true, value : /^aking$/i}

}},
{acl_fields:{$elemMatch:{name:{$ne:"gender"}, public:true, value:/aking/i}}}]})



 Comments   
Comment by Aaron Staple [ 14/Feb/11 ]

Resolving because we have created a separate jira for the root cause.

Comment by Aaron Staple [ 17/Dec/10 ]

Ok, this is a real bug. I've filed SERVER-2245.

Just as a note to myself - the provided example doesn't fail in 1.7 but the underlying problem is still there.

Comment by Amos King [ 17/Dec/10 ]

Change the indexes to this:

db.system.indexes.find()
{ "name" : "id", "ns" : "t1d_development.users", "key" :

{ "_id" : 1 }

}
{ "name" : "email_1_encrypted_password_1", "ns" : "t1d_development.users", "key" :

{ "email" : 1, "encrypted_password" : 1 }

, "unique" : true }
{ "name" : "memberships_1", "ns" : "t1d_development.users", "key" :

{ "memberships" : 1 }

, "unique" : false }
{ "name" : "acl_fields.public_1_acl_fields.circles_1", "ns" : "t1d_development.users", "key" :

{ "acl_fields.public" : 1, "acl_fields.circles" : 1 }

, "unique" : false }
{ "name" : "acl_fields.value_1", "ns" : "t1d_development.users", "key" :

{ "acl_fields.value" : 1 }

, "unique" : false }

Then you will get the duplicates again on 1.6.5

Comment by Aaron Staple [ 17/Dec/10 ]

I was unable to reproduce with 1.6.5 using the provided document, indexes, and query. I believe the failure described is SERVER-1883 which has already been fixed and backported to 1.6.4.

Generated at Thu Feb 08 02:59:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.