For a pipeline of the following shape:
[{$lookup: {let: {a: ...}, pipeline: [{$facet: {array: [{$redact: {...$$a...}}]}}]}]
We can get incorrect results with optimizations enabled if the following occurs:
1. $lookup creates the sequential cache and slaps it at the end of the sub-pipeline
2. $lookup then attempts to re-order the cache to the optimal position with a non-correlated prefix. It relies on the dependency tracker to achieve this.
3. The $facet within the $lookup reports it's dependencies, (incorrectly?) indicating that it does not depend on any of the let variables even though it has a sub-pipeline which refers to $$a. This happens because the $redact stage within the $facet does not support dep tracking, so the variable reference is not found.
4. The cache does not swap with the facet since it believes it to be non-correlated, which can lead to incorrect query results.
Note that in step 3, if a stage within the $facet is "not supported" for dep analysis, it is not propagated back to the facet but instead ends up setting the needWholeDocument flag.
- related to
-
SERVER-63141 Difference in $lookup/$redact/$let behaviour with pipeline optimization
- Closed
-
SERVER-63845 Separate interface to get set of referenced variables from DocumentSource::getDependencies()
- Closed