-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Query Integration
-
None
-
3
-
TBD
-
None
-
None
-
None
-
None
-
None
-
None
-
None
During SERVER-104736 (PR) we refactored the $scoreFusion desugar to use a single 'internal_fields' object to always kept all internal fields used in the desugar in a single top level object. This was originally done to support being able to reference collection level fields in your combination method expression, but also in general creates a more simplified / predictable intermediate document structure that always looks like:
{_id: ..., _internal_scoreFusion_docs: {...}, _internal_scoreFusion_internal_fields: {...}}
The purpose of this ticket is to make $rankFusion's intermediate document structure look the same, and conceptually align the desugars of both stages as much as possible.
This has 2 immediate benefits we can identify:
- query stats: right now query stats are logged after a pipeline is fully desugared and optimized. So for a hybrid search it will be difficult to discern that it was a hybrid search, without these "internal_field" prefixes which clearly label the pipeline is a hybrid search
- more consistent behavior between stages, especially in sharded clusters: there are multiple desugared pipelines we could write that conceptually arrive at the same result. In theory, they should all behave the same way (after re-writes like optimization) but in practice there can be peculiarities of how specific pipelines execute, especially when sharding is involved. By making the stages desugar in a similar way, we will minimize the inconsistencies in their behavior.
- One concrete example of this is when testing upgrade / downgrade behavior, we noticed a difference between $rankFusion / $scoreFusion (rank fusion did something non-ideal in mixed version clusters, whereas score fusion behaved as we would want it to). See if after this desugar is changed if SERVER-108470 becomes effectively resolved and 'whenBinariesAreLatestAndFCVIsLastLTS' can be set to the function where rank fusion is always expected to fail.
- is related to
-
SERVER-108470 Investigate whenBinariesAreLatestAndFCVIsLastLTS inconsistency in rank fusion upgrade downgrade testing
-
- Open
-
-
SERVER-104736 Enable referencing collection level variables in $scoreFusion expression combination option without 'docs.' prefix
-
- Closed
-