[SERVER-16381] Index scans don't include the final "end" check within nscanned/keysExamined Created: 02/Dec/14  Updated: 28/Oct/15  Resolved: 08/May/15

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: 3.1.3

Type: Bug Priority: Minor - P4
Reporter: David Hows Assignee: David Storch
Resolution: Done Votes: 0
Labels: ET
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-17635 Improve SortedDataInterface::Cursor API Closed
Documented
is documented by DOCS-5392 Document change to keysExamined for t... Closed
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Quint Iteration 3
Participants:

 Description   

When scanning an index we will check a key to confirm if we have hit the end. If we have hit the end, we will stop the scan but not include that final index entry scanned within the keysExamined counter.

Leaving the index scan
Incrementing the keysExamined counter

Simple reproducer:

 
for(i=0;i<111;i++){db.t2.insert({x:i})} 
db.t2.ensureIndex({x:1}) 
db.t2.find({x:{$in:[3,50,74,100]}}).explain(true) 

 
.... "keysExamined" : 7, 

This would scan the keys at the x values of 3, 4, 50, 51, 74, 75, 100 and 101. A total of 8 keys, while the counter remains at 7



 Comments   
Comment by Githook User [ 08/May/15 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-16381 last key examined by IXSCAN stage should also count towards keysExamined
Branch: master
https://github.com/mongodb/mongo/commit/df7e94f086f23a4ee9a10679ce7c6cf2ecbb2720

Comment by David Storch [ 07/May/15 ]

We have decided to make this change now, as it falls out of a recent refactor to the IndexScan query execution stage (see db59e0f31b12966).

Comment by David Storch [ 15/Jan/15 ]

In my opinion, this is works as designed. It is indeed the case that the IndexScan stage currently counts each retrieved key towards nscanned, as well as counting each skip between ranges towards nscanned. The final end check is not counted.

That said, the value of nscanned reflects the internal behavior of the index scanning logic. We do not promise to the user that there will be some invariant such as "nscanned is equal to the number of matching keys", or even "nscanned is equal to the number of matching keys plus the number of times that we check for the end of a contiguous scan range". The docs are appropriately vague:

nscanned is the total number of index entries scanned (or documents for a collection scan).

The order of magnitude of the nscanned value is much more important than precisely what the number is. A user probably shouldn't have to care whether or not nscanned is 4 instead of 3, but should definitely be concerned if it is 100 instead of 10.

Although I see the point that perhaps the final end check should also be counted, I am unsure if the argument is strong enough to require a behavior change in this area. Closing this ticket as Works as Designed for now, but of course happy to re-open for further debate or if there are any further concerns

Best,
Dave

Generated at Thu Feb 08 03:40:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.