[SERVER-18027] Support aggregation $out to sharded collection Created: 14/Apr/15  Updated: 04/Sep/18  Resolved: 28/Aug/18

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 4.1.3

Type: New Feature Priority: Major - P3
Reporter: Jon Rangel (Inactive) Assignee: Kyle Suarez
Resolution: Done Votes: 0
Labels: usability
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-35895 Add ability for $out to write to remo... Closed
depends on SERVER-35954 Build uniqueKey from the shard key if... Closed
depends on SERVER-36123 Reject $out with mode: "replaceCollec... Closed
depends on SERVER-12280 allow $out to append to/merge with ex... Closed
is depended on by SERVER-6025 Similar mysql syntax "INSERT ... SELE... Closed
is depended on by SERVER-36114 Test sharded $out concurrently execut... Closed
Documented
is documented by DOCS-12024 Docs for SERVER-18027: Support aggreg... Closed
Duplicate
is duplicated by SERVER-6025 Similar mysql syntax "INSERT ... SELE... Closed
Related
related to SERVER-35897 Support dropTarget false in $out Closed
Backwards Compatibility: Major Change
Sprint: Query 2018-08-27, Query 2018-09-10
Participants:
Case:

 Description   

For aggregations in a sharded cluster, it would be useful for the $out pipeline operator to support output to a sharded collection, to support use cases where the result set of the aggregation is large.



 Comments   
Comment by Kyle Suarez [ 28/Aug/18 ]

We've made many improvements to the $out aggregation stage in the 4.1 development release. Two important pieces of infrastructure have been completed:

  • SERVER-35895: $out can write to remote hosts
  • SERVER-12280: $out can merge its output with an existing collection, rather than replacing it outright

With these in place, users can now use the $out stage to output to an existing sharded collection. This is supported with new $out modes, "insertDocuments" and "replaceDocuments", which will use inserts and replacement-style updates, respectively, to perform writes to the output collection without dropping it.

We won't permit a $out to a sharded output collection if the mode is "replaceCollection", which matches the original $out behavior in MongoDB 4.0 and earlier.

Users can try out sharded $out aggregations by checking out a development release, though the feature is still under active development. We'll release documentation with more thorough details when the feature work is complete.

Regards,
Kyle

Generated at Thu Feb 08 03:46:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.