Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-69188

Disable use of SBE on the inner side of DocumentSourceLookup when the inner side is sharded

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution

      This ticket proposes an extension of SERVER-69103 which prevents SBE from being used on the inner side of a DocumentSourceLookup when the lookup is performed locally. This is implemented using a flag on the ExpressionContext which is not serialized across the wire. Therefore, it is still possible to use SBE on the inner side of a $lookup when the inner side is sharded – the shard receives a query across the wire with no indication that it is executing as part of a $lookup.

      We may want to force the use of the classic engine in this scenario for the same reasons that we chose to implement SERVER-69103, performance chief among them. This would require us to serialize the inLookup flag to the aggregation request, which I believe could be done in the genericTransformForShards() function.

      One challenge with this patch will be to ensure that it works correctly in a multiversion cluster. We cannot unconditionally serialize the inLookup flag, since in something like a 6.0/7.0 mixed version cluster the 6.0 nodes will reject a command containing this new flag. We could look into guarding the addition of the flag based on the featureCompatibilityVersion, but I worry that this wouldn't work correctly if a sharded $lookup is running concurrently with the setFCV() command. One solution would be to propagate some state from mongos (which is always upgraded last) which indicates that the cluster is fully upgraded, and it is safe to serialize the inLookup flag.

            backlog-query-execution [DO NOT USE] Backlog - Query Execution
            david.storch@mongodb.com David Storch
            0 Vote for this issue
            5 Start watching this issue