-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Query Integration
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Overview: This work introduces $_internalDocumentResultsAndMetadata, a new internal agg stage that serves as a transport container for source stages (e.g. $extensionSearch) that emit both a document-results stream and a metadata stream. It also lays the exec-layer foundation for 1:N DocumentSource-to-exec-stage translation.
Scope of Work
$_internalDocumentResultsAndMetadata stage skeleton.
* New DocumentSource that wraps a source stage and fans its output into two consumer pipelines via DocumentSourceExchange.
* Document stream flows into the top-level pipeline.
* Metadata stream bound to a named pipeline variable (e.g. $$SEARCH_META) via $setVariableFromSubPipeline
* On sharded queries, router serializes the stage with returnCursor: true: metadata returns as a secondary cursor, and the router injects $setVariableFromSubPipeline into the merging pipeline.
* Includes explain, query stats, and auth support.
Metadata elision optimization
* optimizeAt() removes the metadata consumer pipeline when no downstream stage references $$SEARCH_META, avoiding unnecessary exchange overhead.
1:N exec stage registry
* Extend exec::agg DocumentSourcetoStage registry to support 1:N mappings.
* Allows a single DocumentSource to translate into multiple exec stages.
Translation function for $_internalDocumentResultsAndMetadata
* Implement the translation function that produces N exec stages for $_internalDocumentResultsAndMetadata using the new 1:N registry.
Acceptance Criteria
* $_internalDocumentResultsAndMetadata parses, serializes, and round-trips correctly in both standalone and sharded configurations.
* $$SEARCH_META is correctly populated when the metadata field is present and referenced downstream.
* Metadata consumer pipeline is elided when $$SEARCH_META is not referenced by any downstream stage.
* exec::agg registry correctly handles 1:N mappings; existing 1:1 mappings are unaffected.
* Translation of $_internalDocumentResultsAndMetadata produces the correct exec stage graph.
* Auth, explain, and query stats paths all handle the new stage correctly.
* All new unit tests pass; no regressions in existing test suites.
- is depended on by
-
SERVER-126007 Implement metadata elision optimization for $_internalDocumentResultsAndMetadata
-
- In Code Review
-
-
SERVER-126011 [M1 - Search Extension] Update $$SEARCH_META validation helpers to recognize $_documentResultsAndMetadata
-
- Closed
-
-
SERVER-126015 [M1 - Search Extension] Implement create_document_results_and_metadata host service
-
- Closed
-
-
SERVER-126008 [M1 - Search Extension] Implement $_documentResultsAndMetadata compilation and standalone execution workflow
-
- Backlog
-