-
Type:
Bug
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: 3.5.8
-
Component/s: Aggregation Framework
-
Labels:
-
Fully Compatible
-
ALL
-
Query 2017-06-19, Query 2017-08-21, Query 2017-12-18, Query 2018-01-01, Query 2018-01-15, Query 2018-01-29
-
(copied to CRM)
When there is a limit that can be absorbed into the $cursor stage of the pipeline, this limit is not respected when reporting explain execution stats. Consider the following example:
> db.c.drop() true > db.c.createIndex({a: 1}) { "createdCollectionAutomatically" : true, "numIndexesBefore" : 1, "numIndexesAfter" : 2, "ok" : 1 } > for (var i = 0; i < 10; i++) { db.c.insert({a: 1}); } WriteResult({ "nInserted" : 1 }) > db.c.explain("executionStats").aggregate([{$match: {a: 1}}, {$limit: 3}]) { "stages" : [ { "$cursor" : { "query" : { "a" : 1 }, "limit" : NumberLong(3), "queryPlanner" : { "plannerVersion" : 1, "namespace" : "test.c", "indexFilterSet" : false, "parsedQuery" : { "a" : { "$eq" : 1 } }, "winningPlan" : { "stage" : "FETCH", "inputStage" : { "stage" : "IXSCAN", "keyPattern" : { "a" : 1 }, "indexName" : "a_1", "isMultiKey" : false, "multiKeyPaths" : { "a" : [ ] }, "isUnique" : false, "isSparse" : false, "isPartial" : false, "indexVersion" : 2, "direction" : "forward", "indexBounds" : { "a" : [ "[1.0, 1.0]" ] } } }, "rejectedPlans" : [ ] }, "executionStats" : { "executionSuccess" : true, "nReturned" : 10, "executionTimeMillis" : 3, "totalKeysExamined" : 10, "totalDocsExamined" : 10, "executionStages" : { "stage" : "FETCH", "nReturned" : 10, "executionTimeMillisEstimate" : 0, "works" : 11, "advanced" : 10, "needTime" : 0, "needYield" : 0, "saveState" : 1, "restoreState" : 1, "isEOF" : 1, "invalidates" : 0, "docsExamined" : 10, "alreadyHasObj" : 0, "inputStage" : { "stage" : "IXSCAN", "nReturned" : 10, "executionTimeMillisEstimate" : 0, "works" : 11, "advanced" : 10, "needTime" : 0, "needYield" : 0, "saveState" : 1, "restoreState" : 1, "isEOF" : 1, "invalidates" : 0, "keyPattern" : { "a" : 1 }, "indexName" : "a_1", "isMultiKey" : false, "multiKeyPaths" : { "a" : [ ] }, "isUnique" : false, "isSparse" : false, "isPartial" : false, "indexVersion" : 2, "direction" : "forward", "indexBounds" : { "a" : [ "[1.0, 1.0]" ] }, "keysExamined" : 10, "seeks" : 1, "dupsTested" : 0, "dupsDropped" : 0, "seenInvalidated" : 0 } } } } } ], "ok" : 1 }
Here the limit of 3 is indeed present in the $cursor.limit field. However, the $cursor.executionStats field shows that the index scan stage examined all 10 keys and that the fetch stage subsequently retrieved all 10 documents. This reporting is incorrect, since the limit of 3 will prevent us from having to look at all matching keys/documents.
The root cause of this problem is that the limit is applied here as the DocumentSourceCursor is retrieving results from the underlying PlanExecutor. However, this code is not exercised by the explain path. Instead, when running an explain we execute the plan and retrieve stats here. Note that this only affects versions 3.5.5 and newer, e.g. versions after execution stats were added to aggregation explain in SERVER-19758.
To fix this, we should have the DocumentSourceCursor to push the limit down into the PlanExecutor itself. This is also wise from an architecture point of view, since DocumentSourceCursor is intended to be a wrapper around a PlanExecutor rather than acting as a query execution operator in its own right.
- is duplicated by
-
-
- Closed
-
- is related to
-
SERVER-19758 Add "executionStats" and "allPlansExecution" explain modes to aggregation explain
-
- Closed
-
- related to
-
-
- Closed
-
-
-
- Closed
-