[SERVER-82298] Push $LU aggregation macro stage down to SBE Created: 18/Oct/23  Updated: 25/Jan/24

Status: In Code Review
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Kevin Cherkauer Assignee: Kevin Cherkauer
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:

 Description   

$LU is shorthand for a combined $lookup-$unwind stages where $unwind immediately unwinds the output match array of the $lookup. These are created by the optimizer when it sees this stage pairing. The information about the $unwind stage is added to the $lookup stage, and the $unwind stage is then deleted from the aggregation pipeline. The result is a stage that shows as a $lookup in explain output but performs the functionality of both stages, such that results are like those of a SQL join: a single record per match with one match in each, rather than normal $lookup output where all RHS matches for a given LHS document are placed into a single array in a single result document for that LHS document. This optimization avoids materializing the array of all matches (which could even be too large to materialize given BSON object size limits, which means without the optimization the query might actually fail).

 

Planner Updates

Update planner to push down $LUs in eligible prefixes.
 

Stage Builder Updates

Update stage builder to build pushed-down $LU stages.

 
SBE Plan Cache

Encode pushed down $LU stages in the plan cache key (canonical_query_encoder.cpp encodePipeline()).'

 
Update Existing Jstests

Many tests examine explain output, some instances of which will change when $LU is pushed down so will need alternate expected results based on featureFlagSbeFull during development. The old expected results will be removed along with featureFlagSbeFull when the feature is complete.


Generated at Thu Feb 08 06:48:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.