Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-63845

Separate interface to get set of referenced variables from DocumentSource::getDependencies()

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 6.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • Labels:
    • Fully Compatible
    • QO 2022-05-30, QO 2022-06-13, QO 2022-06-27, QO 2022-07-11, QO 2022-07-25, QO 2022-08-08, QO 2022-08-22
    • 6

      The DocumentSource::getDependencies() interface currently returns two pieces of information:

      1. The set of depended-on field paths. This is used to internally generate a projection which can result in covered plans, or can prevent downstream pipeline stages from having to process documents containing unnecessary fields.
      2. The set of referenced variables. At the moment, this is used only to identify non-correlated $lookup sub-pipeline prefixes, so that the results from such a prefix can be cached and re-used.

      Dependency analysis and getting the set of referenced variables are logically different operations, so it makes sense to separate them. Also, it is legal for a DocumentSource to return DepsTracker::NOT_SUPPORTED in order to indicate that it does not participate in dependency analysis – this makes sense for "source" stages such as $cursor or $mergeCursors. However, NOT_SUPPORTED can be problematic for callers which want to analyze which variables are referenced, since they must defensively assume that any variable could be referenced. See the fix from SERVER-63141 for an example.

      In order to make this more natural for callers which want to analyze variable references, we should change getDependencies() so that it no longer returns the set of referenced variables inside the DepsTracker. In its place we should introduce a new virtual method DocumentSource::getReferencedVars(). This would be a pure virtual function that all DocumentSource derived classes must implement. It would return a set of Variables::Id identifiers.

      As part of this work, we should change the $$SEARCH_META static analysis from SERVER-60800 to consume the new DocumentSource::getReferencedVars() interface. As of this writing, this code has not landed in the enterprise modules, but the current plan is for it to have an allowlist of stages that don't support dependency analysis but are known to not have any variable references. The addition of DocumentSource::getReferencedVars() would allow us to replace this workaround with a more easily maintainable solution.

            nicholas.zolnierz@mongodb.com Nicholas Zolnierz
            david.storch@mongodb.com David Storch
            0 Vote for this issue
            9 Start watching this issue