[SERVER-13890] Index bounds builder constructs invalid bounds for multiple negations joined by an $or Created: 09/May/14  Updated: 11/Jul/16  Resolved: 09/May/14

Status: Closed
Project: Core Server
Component/s: Index Maintenance, Querying
Affects Version/s: 2.6.0, 2.6.1
Fix Version/s: 2.6.2, 2.7.1

Type: Bug Priority: Major - P3
Reporter: Ranjith Ramachandra Assignee: David Storch
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Tested
Operating System: ALL
Backport Completed:
Steps To Reproduce:

> db.t.drop();
> db.t.ensureIndex({a: 1});
> db.t.find({$or: [{a: {$ne: 3}}, {a: {$ne: 4}}]});

Participants:

 Description   
Issue Status as of May 15, 2014

ISSUE SUMMARY
Queries with multiple negations over the same field joined by an $or fail when an index is present on that field, because wrong index bounds are being computed. For example:

db.t.ensureIndex({a: 1})
db.t.find({$or: [{a: {$ne: 3}}, {a: {$ne: 4}}]})

The above query is always true and should return all documents, but does not.

USER IMPACT
Queries of this type fail with an assertion error.

WORKAROUNDS
Whenever possible, rewrite these queries to avoid having multiple negations over the same field.

AFFECTED VERSIONS
Production releases 2.6.0 and 2.6.1 are affected by this issue.

FIX VERSION
The fix is included in the 2.6.2 production release.

RESOLUTION DETAILS
The problem occurs because the wrong index bounds are computed in the presence of multiple negations on the same field joined by an $or. Version 2.6.2 constructs the correct index bounds.

Original description

I know the condition itself does not make much sense. But there is a bug nevertheless.

To recreate:

$ wget http://hci.stanford.edu/jheer/workshop/data/worldbank/worldbank.csv
$ mongoimport -d test --file worldbank.csv --type csv --headerline
$ mongo test
MongoDB shell version: 2.6.0
connecting to: test
> db.worldbank.findOne()
{
	"_id" : ObjectId("536cb371267a41551a350ce0"),
	"Country" : "Belarus",
	"Year" : 2000,
	"CO2 emissions (metric tons per capita)" : 5.91,
	"Electric power consumption (kWh per capita)" : 2988.71,
	"Energy use (kg of oil equivalent per capita)" : 2459.67,
	"Fertility rate, total (births per woman)" : 1.29,
	"GNI per capita, Atlas method (current US$)" : "1.38E+03",
	"Internet users (per 1,000 people)" : 18.69,
	"Life expectancy at birth, total (years)" : 68.01,
	"Military expenditure (% of GDP)" : 1.26,
	"Population, total" : "1.00E+07",
	"Prevalence of HIV, total (% of population ages 15-49)" : ""
}
> db.worldbank.find({$or: [{"CO2 emissions (metric tons per capita)": {$ne:6}}, {"CO2 emissions (metric tons per capita)":{$ne:5}}]}).count()
1362
> db.worldbank.createIndex({"CO2 emissions (metric tons per capita)":1})
{
	"createdCollectionAutomatically" : false,
	"numIndexesBefore" : 1,
	"numIndexesAfter" : 2,
	"ok" : 1
}
> db.worldbank.find({$or: [{"CO2 emissions (metric tons per capita)": {$ne:6}}, {"CO2 emissions (metric tons per capita)":{$ne:5}}]}).count()
2014-05-09T16:24:45.777+0530 count failed: {
	"code" : 0,
	"ok" : 0,
	"errmsg" : "0 assertion src/mongo/db/query/index_bounds_builder.cpp:843"
} at src/mongo/shell/query.js:191

Another similar error that occurred for the same reasons: This happened in pymongo.

OperationFailure: command SON([('count', u'DS_1'), ('fields', {u'batch_id': 0, u'_id': 0, u'sequence_number': 0}), ('query', {u'$and': [{u'$and': [{u'$or': [{u'column_5': {u'$options': u'-i', u'$regex': u'^(?!catefory_sale$)'}}, {u'column_5': {u'$options': u'-i', u'$regex': u'^(?!category_disCount$)'}}]}, {u'$or': [{u'column_9': {u'$ne': 0}}, {u'column_9': {u'$ne': 1}}]}]}, {}]})]) failed: exception: assertion src/mongo/db/query/plan_enumerator.cpp:1011



 Comments   
Comment by Billy Tetrud [ 25/Jan/16 ]

I'm still using version 2.4.0, but I'm planning on updating in the next couple days because I'm running across this bug. I'm hopeful that because the issue in this ticket was fixed that my issue will be too.

Comment by Ramon Fernandez Marina [ 25/Jan/16 ]

fresheneesz, I'm not able to trigger an assertion error using your example. What version of MongoDB are you using and what's the exact behavior you're seeing? If you could open a new SERVER ticket with this information it would make it easier to investigate further.

Thanks,
Ramón.

Comment by Billy Tetrud [ 23/Jan/16 ]

For anyone still on an old version, its not just multiple negations that triggered this problem. I also got it with something like this:

db.t.find({$and:[
   {$or:[{_id:{$in:[1,2,3]}},{_id:{$exists:false}}]},
   {$or:[{a:'awef'},{a:{$exists:false}}]}    
]})

Comment by Atul Kachru [ 05/Aug/14 ]

CAP ticket added.

Comment by Githook User [ 16/May/14 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-13890 index bounds builder should not pass already-constructed intervals to translate()
(cherry picked from commit d19801d6209d6635e4fb9c42245723f1f1869bf0)
Branch: v2.6
https://github.com/mongodb/mongo/commit/f076d57d427e7fde945303aaec32ba594de5872c

Comment by Githook User [ 09/May/14 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-13890 index bounds builder should not pass already-constructed intervals to translate()
Branch: master
https://github.com/mongodb/mongo/commit/d19801d6209d6635e4fb9c42245723f1f1869bf0

Comment by David Storch [ 09/May/14 ]

Hi ranjith19, thanks for reporting this bug. I have confirmed that this issue still exists, and can reproduce it locally using these steps:

> db.t.drop();
> db.t.ensureIndex({a: 1});
> db.t.find({$or: [{a: {$ne: 3}}, {a: {$ne: 4}}]});

The problem is related to how we construct index bounds in the presence of multiple negations joined by an $or.

Comment by Ranjith Ramachandra [ 09/May/14 ]

Sorry for not formatting the issue correctly. But I could not figure how to do it

Generated at Thu Feb 08 03:33:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.