[SERVER-14972] Cursor count incorrectly reports non-zero value Created: 11/Aug/14  Updated: 16/May/15  Resolved: 16/May/15

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Don Mitchell Assignee: Sam Kleinman (Inactive)
Resolution: Incomplete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

mongodb-2.2, pymongo 2.4,


Attachments: Text File mongo_index_bug.txt    
Operating System: ALL
Participants:

 Description   

(I'm marking this minor due to the ancientness of the release not severity of the issue.)

We created a new index and suddenly queries using that index were reporting non-zero counts when they had zero elements in their resultset. I've put the console output in the attachment which shows cursor.count() returning non-zero but attempts to access element 0 generating an error and then shows the explain which clearly shows that `n` is 0.

The index we created was

ensureIndex({'_id.tag': 1, '_id.org': 1, '_id.course': 1, 'definition.children': 1})

When we deleted the index, the code worked once again.

This is for the open sourced edx.org code base (feel free to look at the code base.py _get_raw_parent_location)



 Comments   
Comment by Sam Kleinman (Inactive) [ 29/Apr/15 ]

Sorry for the delay here, I hope we can get to the bottom of this

There have been some substantial changes to indexes since the 2.2 series (which is EOL,) and there were some improvements to the changes in the 2.4 series that seem related to your case.

Most importantly, earlier versions (before 2.6) of MongoDB did not strictly enforce the 1024 byte index key length restriction, which lead to indexes that omitted documents with longer keys. (e.g. SERVER-10749 and SERVER-3372) If the indexed fields don't exist in all the documents you're counting, this would easily explained the observed behavior. This effect is part of the issue addressed in SERVER-10435.

Otherwise, you may be observing SERVER-4529 if some indexed fields don't exist in all documents. The correctness issues with count operations were fixed in 2.4 and better indexed field length validation was added in 2.6.

  • Do you have plans to upgrade to a more recent version of MongoDB?
  • Just to verify when you say "we deleted the index, the code worked again," mean that the count reported zero (correctly) or the query returned results?
  • Do you have a way to validate your code and data against a 2.6 or a 3.0 release of MongoDB?

Cheers,
sam

Comment by Don Mitchell [ 20/Aug/14 ]

No, we should shard it, but we haven't. (and yes, when I noticed this ticket was on doc I wondered how and why it was but didn't fix it)

Comment by Thomas Rueckstiess [ 20/Aug/14 ]

Hi Don,

I moved this ticket over from the documentation project to SERVER, where we handle bugs and feature requests for MongoDB.

Is this on a sharded environment? If so, then you may be running into SERVER-3645. Counts are currently not using the chunk managers on the primaries, which would filter out the documents not "owned" by a shard. That means you can get counts that are higher than expected.

Thanks,
Thomas

Generated at Thu Feb 08 03:36:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.