-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Query Integration
-
None
-
3
-
TBD
-
None
-
None
-
None
-
None
-
None
-
None
-
None
In the file 'rank_fusion_upgrade_downgrade.js' when running 'testPerformUpgradeDowngradeSharded', the mixed cluster scenario 'whenBinariesAreLatestAndFCVIsLastLTS' should reject all rank fusion queries - however they are currently accepted.
For score fusion, this works as expected where these queries are rejected 'whenBinariesAreLatestAndFCVIsLastLTS' in a sharded cluster.
For this ticket, investigate why these rank fusion queries are accepted when they shouldn't be, and hopefully find and put up a fix so both hybrid search stages behave as expected with upgrade / downgrade.
There is a possibly related or separate issue that right now 'whenBinariesAreLatestAndFCVIsLastLTS' (in a sharded cluster) the specific expected passing command:
const rankFusionPipelineWithScoreDetails = [ { $rankFusion: { input: {pipelines: {barField: [{$sort: {bar: -1}}], field: [{$sort: {foo: 1}}]}}, scoreDetails: true } }, {$project: {scoreDetails: {$meta: "scoreDetails"}, score: {$meta: "score"}}} ]; assert.commandWorked(db.runCommand( {aggregate: collName, pipeline: rankFusionPipelineWithScoreDetails, cursor: {}}));
Works when $rankFusion has one input pipeline, but fails when there are two input pipelines as shown (complaining that 'score' / 'scoreDetails' metadata cannot be projected - meaning the featureFlagRankFusionFull check is failing). We were able to debug that the root of this is that when the $rankFusion has multiple input pipelines, and the $unionWith is present in the desugared pipeline, the router part of the pipeline is actually issued down and executed on a shard, and on a shard the FCV checks are enforced, but on the router they are not - explaining the discrepancy.
We also observed that for multiple input pipelines in this mixed version sharded cluster 'whenBinariesAreLatestAndFCVIsLastLTS' the results produced are out of order. So on multiple fronts its clear that $rankFusion is not interfacing properly with mixed version clusters 'whenBinariesAreLatestAndFCVIsLastLTS'.
Regardless of this is the same, or a separate issue, both issues may end up being resolved by changing the $rankFusion desugaring to look more like $scoreFusion, as $scoreFusion is presenting neither problem. The second problem may also be moot once expecting failure 'whenBinariesAreLatestAndFCVIsLastLTS' for rank fusion, as this problem only arises when expecting success in these sharded cluster cases.
Note, its possible that the effort defined by WRITING-32836 (merging $rankFusion and $scoreFusion into a single internal stage) may end up resolving this issue - as right now $scoreFusion behaves correctly, but $rankFusion does not. So whatever it is about $scoreFusion may get carried over into the implementation of $rankFusion when their implementations are consolidated.
- related to
-
SERVER-108662 Bring $rankFusion desugaring in-line with $scoreFusion (use single 'internal_fields' object)
-
- In Code Review
-