[SERVER-67377] [6.0] Check whether $lookup pushdown feature availability on primary shard in mixed version cluster Created: 17/Jun/22  Updated: 29/Oct/23  Resolved: 28/Jul/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.0.1

Type: Bug Priority: Major - P3
Reporter: Yoon Soo Kim Assignee: Rui Liu
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.0
Sprint: QE 2022-06-27, QE 2022-07-11, QE 2022-07-25, QE 2022-08-08
Participants:
Linked BF Score: 10

 Description   

index_stats.js is failing on sharding_jscore_passthrough_last_continuous and sharding_jscore_passthrough_last_lts test suites because of unreliable feature detection logic of checkSBEEnabled().

Instead of relying on checkSBEEnabled(), better to detect feature availability based on explain result for index_stats.js.

Proposed changes:

// Figure out whether the $lookup with foreign field index is pushed down to the SBE as INLJ.
//
// This function assumes that  'pipeline' has a $lookup stage.
//
// 'checkSBEEnabled(..., featureFlag) does not work very well for a mixed version cluster because
// when it detects the first shard that the 'featureFlag' is turned on, it just returns true but
// in a mixed version cluster, some shard may not support such 'featureFlag'. So, need to check
// whether the $lookup is pushed down or not based on 'explain' result.
var isLookupPushedDownToSBEAsINLJ = function(coll, pipeline) {
    const explainDoc =
        db.runCommand({aggregate: coll.getName(), pipeline: pipeline, explain: true});
    // If $lookup is pushed down to the SBE, 'explainVersion' must be "2".
    if (explainDoc.explainVersion != "2") {
        return false;
    }
    const eqLookupStage = getPlanStage(explainDoc, "EQ_LOOKUP");
    // If $lookup is pushed down to the SBE, EQ_LOOKUP stage must exist.
    if (!eqLookupStage) {
        return false;
    }
    return eqLookupStage.strategy == "IndexedLoopJoin";
};

...

if (!isLookupPushedDownToSBEAsINLJ(col, pipeline)) {
    assert.eq(2,
              getUsageCount("_id_", foreignCollection),
              "Expected each lookup to be tracked as an index use");
} else {
    assert.eq(1,
              getUsageCount("_id_", foreignCollection),
              "Expected the index join lookup to be tracked as a single index use");
}



 Comments   
Comment by Rui Liu [ 22/Jun/22 ]

Discussed offline that we may take a different approach here. `checkSBEEnabled` doesn't work well in the sharded cluster since it returns true when any shard within the cluster returns true. Instead we can run `checkSBEEnabled` on the primary shard instead of mongos.

Generated at Thu Feb 08 06:07:59 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.