Loading...

XML

Word

Printable

JSON

Type: Improvement
Resolution: Done
Priority: Major - P3
Fix Version/s: 3.5.12
Affects Version/s: None
Component/s: Aggregation Framework, Sharding
Labels:
- eng-l
- optimization

Backwards Compatibility:
Fully Compatible
Sprint:
Query 13 (04/22/16), Query 2017-07-10, Query 2017-07-31, Query 2017-08-21

Suppose you have a pipeline consisting of a single $match stage which is delivered to mongos by the client. Also suppose that mongos will target multiple shards in order to find the matching data. Currently, the aggregation system will establish a shard as the merging node. Instead, mongos could perform the merging (it already knows how to do this for .find() operations).

This would improve the latency of the aggregation operation, since it would eliminate a network hop. Data currently has to travel from a targeted shard, to the merging shard, to mongos, and back to the client. If mongos were responsible for merging, data would only have to travel from a targeted shard, to mongos, to the client.

Mongos can also merge sorted streams coming from the shards for .find() operations, so this could also be extended to $match+$sort pipelines.

depends on

SERVER-27937 pull apart the AsyncResultsMerger logic into (1) establishing the cursors (2) sending the getMore's

Closed

is depended on by

SERVER-29141 Add sharding support for merging changeNotification responses from all shards

Closed

is related to

SERVER-18940 Optimise sharded aggregations that are targeted to a single shard

Closed

SERVER-142 Read-only views over collection data.

Closed

DOCS-11083 Create a table of where sorts happen in different server versions

Closed

related to

SERVER-27637 Merging phase of a distributed $sample should recognize input streams are already sorted

Closed

SERVER-30412 mongos Segmentation fault during aggregation workload

Closed

SERVER-32282 Aggregation text search returns text score even if it wasn't requested when targeting multiple shards in a sharded cluster

Closed

SERVER-32297 Aggregations that merge on mongos do not respect the collation

Closed

SERVER-23955 Return what host the Shard::runCommand actually ran on

Backlog

SERVER-30480 Update aggregation explain format to provide details of merge location

Closed

SERVER-30871 Permit blocking aggregation stages to run on mongoS if allowDiskUse is false

Closed

(7 related to)

Assignee:: Bernard Gorman

Reporter:: David Storch

Participants:: Andy Schwerin, Asya Kamsky, Bernard Gorman, Charlie Swanson, David Storch, Githook User, Mathias Stearn

Votes:: 0 Vote for this issue

Watchers:: 15 Start watching this issue

Created:: Feb 19 2016 03:30:24 PM UTC

Updated:: May 20 2021 09:47:23 AM UTC

Resolved:: Aug 08 2017 06:46:36 PM UTC

Details

Description

Attachments

Issue Links

Activity

People

Dates