[Server] Implement $_internalDocumentResultsAndMetadata stage and supporting exec machinery

XMLWordPrintableJSON

    • Query Integration
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Overview: This work introduces $_internalDocumentResultsAndMetadata, a new internal agg stage that serves as a transport container for source stages (e.g. $extensionSearch) that emit both a document-results stream and a metadata stream. It also lays the exec-layer foundation for 1:N DocumentSource-to-exec-stage translation.
       
      Scope of Work
      $_internalDocumentResultsAndMetadata stage skeleton.
        * New DocumentSource that wraps a source stage and fans its output into two consumer pipelines via DocumentSourceExchange.
        * Document stream flows into the top-level pipeline.
        * Metadata stream bound to a named pipeline variable (e.g. $$SEARCH_META) via $setVariableFromSubPipeline
        * On sharded queries, router serializes the stage with returnCursor: true: metadata returns as a secondary cursor, and the router injects $setVariableFromSubPipeline into the merging pipeline.
        * Includes explain, query stats, and auth support.
       
      Metadata elision optimization
        * optimizeAt() removes the metadata consumer pipeline when no downstream stage references $$SEARCH_META, avoiding unnecessary exchange overhead.
       
      1:N exec stage registry 
        * Extend exec::agg DocumentSourcetoStage registry to support 1:N mappings.
        * Allows a single DocumentSource to translate into multiple exec stages.
       
      Translation function for $_internalDocumentResultsAndMetadata
        * Implement the translation function that produces N exec stages for $_internalDocumentResultsAndMetadata using the new 1:N registry.
       
      Acceptance Criteria
        * $_internalDocumentResultsAndMetadata parses, serializes, and round-trips correctly in both standalone and sharded configurations.
        * $$SEARCH_META is correctly populated when the metadata field is present and referenced downstream.
        * Metadata consumer pipeline is elided when $$SEARCH_META is not referenced by any downstream stage.
        * exec::agg registry correctly handles 1:N mappings; existing 1:1 mappings are unaffected.
        * Translation of $_internalDocumentResultsAndMetadata produces the correct exec stage graph.
        * Auth, explain, and query stats paths all handle the new stage correctly.
        * All new unit tests pass; no regressions in existing test suites.

            Assignee:
            Aditya Deshpande
            Reporter:
            Daniel Segel
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: