[SERVER-4665] Querying against sparse index creates bad state, affecting later queries Created: 12/Jan/12 Updated: 11/Jul/16 Resolved: 17/Jan/12 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Index Maintenance, Querying |
| Affects Version/s: | 2.0.1 |
| Fix Version/s: | 2.0.3, 2.1.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Paul Canavese | Assignee: | Aaron Staple |
| Resolution: | Done | Votes: | 2 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Shell and DB running under OSX. |
||
| Attachments: |
|
| Operating System: | OS X |
| Participants: |
| Description |
|
I have run into a case where querying against a sparse index on a nested field puts the DB in a bad state that affects later queries on the same collection. I originally discovered this problem using the ruby driver, but was able to reproduce the problem completely from the mongo console (see below). I inserted comments, prefixed with #. The mongo_bug db did not exist prior to this example. > use mongo_bug , {sparse: true}) ) ) ) > db.bug_collection.find({key: {$in: ['a', 'b', 'c']}, "nested.key2": {$ne: 'd'}})
The order of the queries also matters: > db.bug_collection.dropIndex("nested.key2_1") { "nIndexesWas" : 2, "ok" : 1 }> db.bug_collection.ensureIndex( {"nested.key2": 1}, {sparse: true})
It's as if the queries are considered equivalent and being cached. |
| Comments |
| Comment by auto [ 17/Jan/12 ] |
|
Author: {u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}Message: |
| Comment by auto [ 16/Jan/12 ] |
|
Author: {u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}Message: |
| Comment by auto [ 16/Jan/12 ] |
|
Author: {u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}Message: |
| Comment by auto [ 16/Jan/12 ] |
|
Author: {u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}Message: |
| Comment by Aaron Staple [ 12/Jan/12 ] |
|
Hi Paul, I'll try to give a brief overview of what's happening here. Mongo uses a sampling based mechanism to determine which indexes seem effective for resolving particular queries. But we don't use sampling all the time - we also keep a record of indexes that seem to perform well for particular queries and do query pattern matching to attempt to retry those performant indexes for subsequent queries that are similar, but not identical, to earlier queries. In this case the $ne part of your query is being ignored for pattern matching, so the index that worked for your first query is being reused for your second query. And since that index is sparse, using it can alter the result of the query. We should probably include your $ne field in the pattern matching model, but will need to figure out exactly how to do that. When we decide how to do that, I'll link to the corresponding ticket. |
| Comment by Paul Canavese [ 12/Jan/12 ] |
|
I also should note that everything works fine if the index is not sparse. |
| Comment by Paul Canavese [ 12/Jan/12 ] |
|
The formatting didn't come out well. Uploading the commands/commentary as an attachment. |