[SERVER-16512] totalDocsExamined is counted twice in geo near Created: 11/Dec/14  Updated: 02/Feb/18  Resolved: 15/Jan/15

Status: Closed
Project: Core Server
Component/s: Geo, Querying
Affects Version/s: 2.8.0-rc2
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Siyuan Zhou Assignee: David Storch
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to DOCS-11277 Consider slight clarification of `exp... Closed
Backwards Compatibility: Fully Compatible
Participants:

 Description   

In NearStage, we generate serval sub-stages to fetch and sort the data progressively. Every sub-stage is a fetch stage which has a index scan stage as a child. On top of NearStage, we use another top level fetch stage to filter the data if necessary, so it turns out that the "totalDocsExamined" is counted twice for the returned documents, one in the sub fetch stage and one in the top level fetch stage.

Ideally, "totalDocsExamined" should indicate the number of documents we fetched from disk and "alreadyHasObj" indicates the number of documents already in memory. Currently, "totalDocsExamined" includes "alreadyHasObj", which makes the "totalDocsExamined" greater than expected for geo near.

Another approach is to expose the total number of objects already in memory at the top level to make it explicit that totalDocsExamined is not a metric of disk fetch. Or instead, expose the number of documents fetched from disk directly.



 Comments   
Comment by David Storch [ 15/Jan/15 ]

Closing this as Works as Designed. The double-counting happens because the geoNear plan passes each document through two fetch stages, with two different filters. Each document is indeed examined twice.

For a more detailed explanation, please refer to the comment added in commit e5800d50ed.

Comment by Githook User [ 15/Jan/15 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-16512 add comment explaining 'docsExamined' tracking in the FetchStage
Branch: master
https://github.com/mongodb/mongo/commit/e5800d50ed4152a31630f25b5dffea106a93de07

Generated at Thu Feb 08 03:41:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.