$_internalDocumentResultsAndMetadata loses DPL inside sharded subpipelines

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Integration
    • ALL
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Extension stages that desugar into DRM during LiteParsed expansion carry their DPL as an
      in-memory callback supplied by the extension. This callback is not part of the stage's BSON serialization.

      When a DRM stage is nested inside a $lookup or $unionWith subpipeline that targets a sharded collection, the router serializes the already-desugared subpipeline to BSON and ships it to the shards. The in-memory callback cannot survive the BSON round-trip, so the shard re-parses DRM without it.

      As a result, DRM's distributedPlanLogic() has no plan to offer on the shard:

      • document results are not merge-sorted across shards, and
      • the metadata stream is not merged, so $$SEARCH_META is not populated.

      Top-level (non-subpipeline) sharded DRM is not affected, because the callback reaches the router's DRM stage in-process.

      SERVER-128789 fixed callback loss across the in-process LiteParsed handoff. This ticket covers the remaining case where the callback must cross a BSON serialization boundary into a subpipeline.

            Assignee:
            Daniel Segel
            Reporter:
            Daniel Segel
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: