Consider opening connections to shards only when merge pipeline is executed

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution
    • None
    • 3
    • TBD
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Currently, we open connections to the shards on mongoS in sharded_agg_helpers::dispatchShardPipeline() before the pipeline is executed.

      The open connections are then passed to the DocumentSourceMergeCursors class which is supposed to 'own' them and kill remote cursors when necessary. I am not sure what happens with killing remote cursors before DocumentSourceMergeCursors is created.

      In the context of SPM-4106, ideally, we only need to open shard connections when exec::agg::Pipeline is created, but this requires some refactoring in cluster_aggregation_planner code. This would simplify management of opened connections and also reduce the risk of leaking remote cursors.

      This ticket partially depends on SPM-4106 results, but it is out of the projects scope.

            Assignee:
            Unassigned
            Reporter:
            Romans Kasperovics
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: