-
Type: Task
-
Resolution: Unresolved
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Query Integration
The field 'preserveOrderAndMetadata' on the StageConstraints struct describes, as the name suggests, whether any given stage preserves the ordering of the documents it outputs and preserves the metadata on those documents.
The StageConstraints struct is available for any DocumentSource, and must be implemented by any concrete DocumentSource.
Currently, there are many stages that do not specify that they preserveOrderAndMetadata when they do. We need as many stages as possible properly tagged with this field, so that we can perform a generic optimization related removing $sorts. Specifically, if we know that a stage outputs documents in a sorted order, any number of stages that all preserve the sort order should be able to be placed between that stage and a $sort stage that sorts on the same criteria for the $sort to be safely removed.
This ticket enables previous optimizations in SERVER-96059, SERVER-96064, & SERVER-96065 to be further generalized to more pipeline types.
Were most interested in tagging the most commonly used stages in this PR, and we should aim to be as exhaustive as possible while not blocking on obscure stages. Some stages of interest might be: $match, $limit, $skip, $lookup.
We may also consider introducing a second field that specifics if the stage preserves the order, if the ordering is on metadata. Some stages can't guarantee that the result order will be preserved on a normal field, but could, if the ordering was on a metadata fields like: $project, $addFields, $set, $replaceWith, $unwind.
- is depended on by
-
SERVER-91823 Push down more stages to shards in sharded $search
- Blocked
-
SERVER-96068 Generalize the $sort removal optimizations after $search/$vectorSearch/$mergeCursors
- Open
- is related to
-
SERVER-96065 Optimize away $sort directly after sorted $mergeCursors for sharded environments
- Open
-
SERVER-96059 Optimize away metadata $sort directly after $search for single node environments
- In Code Review
-
SERVER-96064 Optimize away $sort directly after $vectorSearch for single node environments
- In Code Review
-
SERVER-94628 Ensure we do not do a redundant $sort by search score during $rankFusion.
- Closed