[SERVER-27521] For covered index queries in agg, don't hydrate field names before converting to Document Created: 27/Dec/16  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Asya Kamsky Assignee: Backlog - Query Execution
Resolution: Unresolved Votes: 0
Labels: performance
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query Execution
Participants:

 Description   

For covered index queries in agg don't hydrate field names before converting to Document.

		"$cursor" : {
			"query" : {
				"type" : "Music"
				},
				"fields" : {
					"format" : 1,
					"genre" : 1,
					"subclass" : 1,
					"_id" : 0
				},
				"queryPlanner" : {
					"plannerVersion" : 1,
					"namespace" : "bestbuy.products",
					"indexFilterSet" : false,
					"parsedQuery" : {
						"type" : {
							"$eq" : "Music"
						}
					},
					"winningPlan" : {
						"stage" : "PROJECTION",
						"transformBy" : {
							"format" : 1,
							"genre" : 1,
							"subclass" : 1,
							"_id" : 0
						},
						"inputStage" : {
							"stage" : "IXSCAN",
							"keyPattern" : {
								"type" : 1,
								"subclass" : 1,
								"genre" : 1,
								"format" : 1
							},
							"indexName" : "type_1_subclass_1_genre_1_format_1",
							"isMultiKey" : false,
							"multiKeyPaths" : {
								"type" : [ ],
								"subclass" : [ ],
								"genre" : [ ],
								"format" : [ ]
							},
							"isUnique" : false,
							"isSparse" : false,
							"isPartial" : true,
							"indexVersion" : 2,
							"direction" : "forward",
							"indexBounds" : {
								"type" : [
									"[\"Music\", \"Music\"]"
								],
								"subclass" : [
									"[MinKey, MaxKey]"
								],
								"genre" : [
									"[MinKey, MaxKey]"
								],
								"format" : [
									"[MinKey, MaxKey]"
								]
							}
						}
					},
					"rejectedPlans" : [ ]
			}
		}

Ideally this would eliminate "stage" : "PROJECTION" or replace it with agg specific stage.



 Comments   
Comment by Geert Bosch [ 13/Sep/19 ]

I'd like to add that the keys are stored in KeyString format. We shouldn't transform that to BSON (hydrated or not), but straight to document.

Comment by David Storch [ 03/Jan/17 ]

To provide a bit more detail on this request, consider a collection which as a single index {a: 1, b: 1}. In today's system, the following two queries are both covered and both are answered using an IXSCAN=>PROJECTION plan:

db.c.find({a: 1, b: 1}, {a: 1, _id: 0});
db.c.find({a: 1, b: 1}, {a: 1, b: 1, _id: 0});

In the case of the former query, the PROJECTION stage is clearly required: the index keys contain both fields a and b, so we must perform some computation in order to exclude b. In the case of the latter query, the situation is not so obvious. Since we are returning the entire index key, you could imagine that the PROJECTION stage could be left out of the query plan. It is currently required, however, because index keys are stored in a "dehydrated" state, meaning that the field names are absent. The index key for {a: 4, b: 5, c: 6}, for example, would be {'': 4, '': 5}. The PROJECTION stage is used to "rehydrate" keys (i.e. add the field names back in) before returning them to the user.

This ticket tracks a performance optimization in which we could skip the "rehydrating" work for some queries.

Generated at Thu Feb 08 04:15:23 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.