[SERVER-13714] non-top-level indexable $not triggers query planning bug Created: 24/Apr/14  Updated: 11/Jul/16  Resolved: 24/Apr/14

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: 2.6.0
Fix Version/s: 2.6.2, 2.7.0

Type: Bug Priority: Major - P3
Reporter: Raul E Rangel Assignee: hari.khalsa@10gen.com
Resolution: Done Votes: 0
Labels: query
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
Operating System: ALL
Backport Completed:
Steps To Reproduce:

> db.posts.find({"$and":[{"$or":[{"is_draft":false},{"creator_id":ObjectId("5258540936a42b1240000030")}]},{"$or":[{"state":3,"is_draft":false},{"published_date":{"$ne":null}}]},{"newsroom_id":{"$in":[ObjectId("4e962c5bcbcf1f0001000373")]}}]})
 
error: { "$err" : "assertion src/mongo/db/query/plan_enumerator.cpp:1040" }

Participants:

 Description   
Issue Status as of May 14, 2014

ISSUE SUMMARY
A query with a non-top-level $not or $ne operator using an index triggers a bug in the query planning system and fails with an assertion error.

Example:

db.coll.find( {'$or': [ {'state': 1, 'is_draft': 1}, 
                        {'published_date': {'$ne': 1} } ] } )

This query has a negation underneath an $or parent node. If each of the fields is indexed, this query would trigger the bug.

USER IMPACT
Queries matching the above condition fail with an assertion.

WORKAROUNDS
Rewrite the query to avoid non-top-level negation on indexed fields if possible.

AFFECTED VERSIONS
Versions 2.6.0 and 2.6.1 are affected by this issue.

FIX VERSION
The fix is included in the 2.6.2 production release.

RESOLUTION DETAILS
A bug in creating the memoized data structure used to enumerate the space of query plans is now fixed.

Original description

A query that used to work on 2.2 is now failing when upgrading to 2.6.



 Comments   
Comment by Githook User [ 15/May/14 ]

Author:

{u'username': u'hkhalsa', u'name': u'Hari Khalsa', u'email': u'hkhalsa@10gen.com'}

Message: SERVER-13714 fix enumeration of non-top-level indexed negations
(cherry picked from commit 38711ac013df44b1bc928f868ee45a9ac93b3ffb)
Branch: v2.6
https://github.com/mongodb/mongo/commit/faaffb7477e70be293f0e1141f4fff28ffaa2771

Comment by Raul E Rangel [ 24/Apr/14 ]

Awesome thanks!

Comment by hari.khalsa@10gen.com [ 24/Apr/14 ]

ismell the fix has been pushed to the 2.7 branch. It will be back-ported to the next eligible 2.6 release, probably 2.6.2. Thanks for reporting and apologies for the bug. And please continue to file tickets for any other problems you encounter.

Comment by Githook User [ 24/Apr/14 ]

Author:

{u'username': u'hkhalsa', u'name': u'Hari Khalsa', u'email': u'hkhalsa@10gen.com'}

Message: SERVER-13714 fix enumeration of non-top-level indexed negations
Branch: master
https://github.com/mongodb/mongo/commit/38711ac013df44b1bc928f868ee45a9ac93b3ffb

Comment by hari.khalsa@10gen.com [ 24/Apr/14 ]

I've reproduced it locally. It's not exactly $or related but instead a problem with how we generate plans for $not queries that can use an index. The query

{$or:[{state:1, is_draft:1}, {published_date:{$ne: 1}}]}

with indices over each of the three involved fields will also trigger the same bug. I will have a fix soon.

Comment by hari.khalsa@10gen.com [ 24/Apr/14 ]

ismell Thanks for the indices. That combined with the query should be enough for us to reproduce the issue. We'll update this ticket when we have a repro and fix.

Comment by Raul E Rangel [ 24/Apr/14 ]

This happened on OSX using brew install mongodb. I can't seem to change the report.
me@Rauls-MacBook-Pro:~/Projects/napa$ mongod --version
db version v2.6.0
2014-04-24T10:51:58.597-0600 git version: nogitversion

Comment by Raul E Rangel [ 24/Apr/14 ]

Apparently I have a lot of them.

> db.posts.getIndexes()
[
	{
		"v" : 1,
		"key" : {
			"_id" : 1
		},
		"ns" : "napa.posts",
		"name" : "_id_"
	},
	{
		"v" : 1,
		"key" : {
			"invitee_ids" : 1
		},
		"ns" : "napa.posts",
		"name" : "invitee_ids_1",
		"background" : true
	},
	{
		"v" : 1,
		"key" : {
			"monitor_social_media_activity" : 1
		},
		"ns" : "napa.posts",
		"name" : "monitor_social_media_activity_1"
	},
	{
		"v" : 1,
		"key" : {
			"updated_at" : 1
		},
		"ns" : "napa.posts",
		"name" : "updated_at_1"
	},
	{
		"v" : 1,
		"key" : {
			"creator_id" : 1
		},
		"ns" : "napa.posts",
		"name" : "creator_id_1"
	},
	{
		"v" : 1,
		"key" : {
			"newsroom_id" : 1
		},
		"ns" : "napa.posts",
		"name" : "newsroom_id_1"
	},
	{
		"v" : 1,
		"key" : {
			"imported_feed_url" : 1
		},
		"ns" : "napa.posts",
		"name" : "imported_feed_url_1"
	},
	{
		"v" : 1,
		"key" : {
			"categories_array" : 1
		},
		"ns" : "napa.posts",
		"name" : "categories_array_1"
	},
	{
		"v" : 1,
		"key" : {
			"tags_array" : 1
		},
		"ns" : "napa.posts",
		"name" : "tags_array_1"
	},
	{
		"v" : 1,
		"key" : {
			"state" : 1
		},
		"ns" : "napa.posts",
		"name" : "state_1"
	},
	{
		"v" : 1,
		"key" : {
			"publish_group_id" : 1
		},
		"ns" : "napa.posts",
		"name" : "publish_group_id_1"
	},
	{
		"v" : 1,
		"key" : {
			"assignee_id" : 1
		},
		"ns" : "napa.posts",
		"name" : "assignee_id_1"
	},
	{
		"v" : 1,
		"key" : {
			"editor_id" : 1
		},
		"ns" : "napa.posts",
		"name" : "editor_id_1"
	},
	{
		"v" : 1,
		"key" : {
			"transaction_id" : 1
		},
		"ns" : "napa.posts",
		"name" : "transaction_id_1"
	},
	{
		"v" : 1,
		"key" : {
			"is_draft" : 1
		},
		"ns" : "napa.posts",
		"name" : "is_draft_1"
	},
	{
		"v" : 1,
		"key" : {
			"privacy" : 1
		},
		"ns" : "napa.posts",
		"name" : "privacy_1"
	},
	{
		"v" : 1,
		"key" : {
			"invited_group_ids" : 1
		},
		"ns" : "napa.posts",
		"name" : "invited_group_ids_1"
	},
	{
		"v" : 1,
		"key" : {
			"listener_ids" : 1
		},
		"ns" : "napa.posts",
		"background" : true,
		"name" : "listener_ids_1"
	},
	{
		"v" : 1,
		"key" : {
			"post_type_id" : 1
		},
		"ns" : "napa.posts",
		"name" : "post_type_id_1"
	},
	{
		"v" : 1,
		"key" : {
			"_slugs" : 1
		},
		"unique" : true,
		"ns" : "napa.posts",
		"name" : "_slugs_1"
	},
	{
		"v" : 1,
		"key" : {
			"next_task_owner_id" : 1
		},
		"ns" : "napa.posts",
		"name" : "next_task_owner_id_1"
	},
	{
		"v" : 1,
		"key" : {
			"publish_deadline_sort" : 1
		},
		"ns" : "napa.posts",
		"name" : "publish_deadline_sort_1"
	},
	{
		"v" : 1,
		"key" : {
			"submission_deadline_sort" : 1
		},
		"ns" : "napa.posts",
		"name" : "submission_deadline_sort_1"
	},
	{
		"v" : 1,
		"key" : {
			"preferred_title_sort" : 1
		},
		"ns" : "napa.posts",
		"name" : "preferred_title_sort_1"
	},
	{
		"v" : 1,
		"key" : {
			"persona_id" : 1
		},
		"ns" : "napa.posts",
		"name" : "persona_id_1"
	},
	{
		"v" : 1,
		"key" : {
			"stage_id" : 1
		},
		"ns" : "napa.posts",
		"name" : "stage_id_1"
	},
	{
		"v" : 1,
		"key" : {
			"campaign_id" : 1
		},
		"ns" : "napa.posts",
		"name" : "campaign_id_1"
	},
	{
		"v" : 1,
		"key" : {
			"slug_history" : 1
		},
		"ns" : "napa.posts",
		"name" : "slug_history_1"
	},
	{
		"v" : 1,
		"key" : {
			"persona_ids" : 1
		},
		"ns" : "napa.posts",
		"name" : "persona_ids_1"
	},
	{
		"v" : 1,
		"key" : {
			"campaign_ids" : 1
		},
		"ns" : "napa.posts",
		"name" : "campaign_ids_1"
	},
	{
		"v" : 1,
		"key" : {
			"stage_ids" : 1
		},
		"ns" : "napa.posts",
		"name" : "stage_ids_1"
	},
	{
		"v" : 1,
		"key" : {
			"comment_ids" : 1
		},
		"ns" : "napa.posts",
		"name" : "comment_ids_1"
	},
	{
		"v" : 1,
		"key" : {
			"next_task_deadline_sort" : 1
		},
		"ns" : "napa.posts",
		"name" : "next_task_deadline_sort_1"
	},
	{
		"v" : 1,
		"key" : {
			"tasks.completed" : 1,
			"tasks.user_id" : 1,
			"tasks.completion_date" : 1
		},
		"ns" : "napa.posts",
		"name" : "tasks.completed_1_tasks.user_id_1_tasks.completion_date_1"
	},
	{
		"v" : 1,
		"key" : {
			"next_task_owner_ids" : 1
		},
		"ns" : "napa.posts",
		"name" : "next_task_owner_ids_1"
	},
	{
		"v" : 1,
		"key" : {
			"external_site_id" : 1
		},
		"ns" : "napa.posts",
		"name" : "external_site_id_1"
	},
	{
		"v" : 1,
		"key" : {
			"tasks.completed" : 1,
			"tasks.completion_date" : 1
		},
		"ns" : "napa.posts",
		"name" : "tasks.completed_1_tasks.completion_date_1"
	},
	{
		"v" : 1,
		"key" : {
			"tasks.task_type_id" : 1,
			"tasks.user_ids" : 1
		},
		"ns" : "napa.posts",
		"name" : "tasks.task_type_id_1_tasks.user_ids_1"
	},
	{
		"v" : 1,
		"key" : {
			"tasks.task_type_id" : 1,
			"tasks.completion_date" : 1
		},
		"ns" : "napa.posts",
		"name" : "tasks.task_type_id_1_tasks.completion_date_1"
	},
	{
		"v" : 1,
		"key" : {
			"tasks.user_ids" : 1
		},
		"ns" : "napa.posts",
		"name" : "tasks.user_ids_1"
	},
	{
		"v" : 1,
		"key" : {
			"tasks.completed" : 1,
			"tasks.user_ids" : 1,
			"tasks.completion_date" : 1
		},
		"ns" : "napa.posts",
		"name" : "tasks.completed_1_tasks.user_ids_1_tasks.completion_date_1"
	},
	{
		"v" : 1,
		"key" : {
			"visible_to_user_ids" : 1
		},
		"ns" : "napa.posts",
		"name" : "visible_to_user_ids_1",
		"background" : true,
		"safe" : true
	},
	{
		"v" : 1,
		"key" : {
			"visible_to_group_ids" : 1
		},
		"ns" : "napa.posts",
		"name" : "visible_to_group_ids_1"
	},
	{
		"v" : 1,
		"key" : {
			"preferred_title_sort" : 1,
			"updated_at" : -1
		},
		"ns" : "napa.posts",
		"name" : "preferred_title_sort_1_updated_at_-1"
	},
	{
		"v" : 1,
		"key" : {
			"published_date" : 1
		},
		"ns" : "napa.posts",
		"name" : "published_date_1"
	},
	{
		"v" : 1,
		"key" : {
			"created_at" : 1
		},
		"ns" : "napa.posts",
		"name" : "created_at_1"
	},
	{
		"v" : 1,
		"key" : {
			"created_at" : 1,
			"created_as" : 1
		},
		"ns" : "napa.posts",
		"name" : "created_at_1_created_as_1"
	},
	{
		"v" : 1,
		"key" : {
			"preferred_title_sort" : -1,
			"updated_at" : -1
		},
		"ns" : "napa.posts",
		"name" : "preferred_title_sort_-1_updated_at_-1"
	},
	{
		"v" : 1,
		"key" : {
			"assignee_id" : -1
		},
		"ns" : "napa.posts",
		"name" : "assignee_id_-1",
		"background" : true,
		"safe" : true
	},
	{
		"v" : 1,
		"key" : {
			"assigned_date_sort" : 1
		},
		"ns" : "napa.posts",
		"name" : "assigned_date_sort_1"
	}
]

Comment by hari.khalsa@10gen.com [ 24/Apr/14 ]

Thanks for the bug report ismell. What indices do you have on the collection where you are running this query?

Generated at Thu Feb 08 03:32:38 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.