Add get_needed_fields to expose downstream field dependencies to extension stages

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.0.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • Query Integration
    • Fully Compatible
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The MongoExtensionPipelineDependencies object is passed to extension source stages (those with requiresInputDocSource=false) via apply_pipeline_suffix_dependencies. It currently exposes three queries: needs_metadata, needs_variable, and needs_whole_document. However, it does not expose which specific document fields the downstream pipeline references.

      This prevents extension source stages (e.g., $_extensionSearch) from building field-limited queries to their external data source—for example, constructing a stored-source projection for mongot that returns only the fields actually consumed by the pipeline. 

      DepsTracker::fields, a set of dotted field-path strings, are computed as part of the returned DepsTracker result from the dep analysis conducted on pipelines with an extension source stage.

      A get_needed_fields function can be added to MongoExtensionPipelineDependenciesVTable that populates a MongoExtensionByteBuf* with a BSON array of dotted field-path strings representing DepsTracker::fields. The output is null when needs_whole_document is true (field enumeration is meaningless in that case). Implement the associated plumbing.

      Add the corresponding unit and integration tests.

            Assignee:
            Daniel Segel
            Reporter:
            Adithi Raghavan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: