[SERVER-3253] aggregation: unsharded support $out - MongoDB

Details

Type: New Feature
Status: Closed
Priority: Major - P3
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 2.5.2
Component/s: Aggregation Framework
Labels:
None

Description

Implement output option for aggregation pipelines

Issue Links

depends on

SERVER-3627 sharded map-reduce output should be parallelized and properly distribute chunks

Closed

SERVER-5932 support cursor based aggregation

Closed

SERVER-8850 $tee command for aggregation framework

Closed

is depended on by

CSHARP-800 Aggregate Command will support $out

Closed

DRIVERS-111 Support $out aggregation pipeline operator

Closed

JAVA-912 Support $out aggregation pipeline operator

Closed

SERVER-447 new aggregation framework

Closed

DOCS-1832 Document $out support

Closed

is duplicated by

SERVER-610 Temp Collections for Non Map-Reduce Queries

Closed

Show 4 more links (3 is depended on by, 1 is duplicated by)

Activity

Comments

Ascending order - Click to sort in descending order

Hide

Permalink

Chris Westin added a comment - Sep 16 2011 11:43:05 AM +00:00

$out either has options, or supports $tee, which has the same options,
but also passes things along, unlike $out

options to have or not have indexes
might want to choose not to have an index if you write to a non-local or
non-temp collection
when we do this, make index creation go after insertion into the collection,
if the collection is new

Show

Chris Westin added a comment - Sep 16 2011 11:43:05 AM +00:00 $out either has options, or supports $tee, which has the same options, but also passes things along, unlike $out options to have or not have indexes might want to choose not to have an index if you write to a non-local or non-temp collection when we do this, make index creation go after insertion into the collection, if the collection is new

15 older comments

Hide

Permalink

auto (Inactive) added a comment - Jul 26 2013 09:56:59 PM +00:00

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: ~~SERVER-3253~~ Unsharded $out
Branch: master
https://github.com/mongodb/mongo/commit/ea776af8e3f982c488ca6d7476ab79e20733e9a9

Show

auto (Inactive) added a comment - Jul 26 2013 09:56:59 PM +00:00 Author: {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'} Message: SERVER-3253 Unsharded $out Branch: master https://github.com/mongodb/mongo/commit/ea776af8e3f982c488ca6d7476ab79e20733e9a9

Hide

Permalink

auto (Inactive) added a comment - Aug 19 2013 11:14:54 PM +00:00

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: Use bulk-insert in $out

About 4x faster at simple copy-collection benchmark.
Follow-up to ~~SERVER-3253~~
Branch: master
https://github.com/mongodb/mongo/commit/f8e37b3b9daffbb721e231bc4418b6c7aac93e8d

Show

auto (Inactive) added a comment - Aug 19 2013 11:14:54 PM +00:00 Author: {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'} Message: Use bulk-insert in $out About 4x faster at simple copy-collection benchmark. Follow-up to SERVER-3253 Branch: master https://github.com/mongodb/mongo/commit/f8e37b3b9daffbb721e231bc4418b6c7aac93e8d

Hide

Permalink

Shane R. Spencer added a comment - Aug 26 2013 11:10:40 PM +00:00

I believe using a temporary collection and renaming it should be optional. I would love to use capped or TTL collections for aggregation results. I'm not sure if making this optional would also allow sharding to be done. I'm not too familiar with where code is processed when mongos is involved.

Show

Shane R. Spencer added a comment - Aug 26 2013 11:10:40 PM +00:00 I believe using a temporary collection and renaming it should be optional. I would love to use capped or TTL collections for aggregation results. I'm not sure if making this optional would also allow sharding to be done. I'm not too familiar with where code is processed when mongos is involved.

Hide

Permalink

Dan Pasette added a comment - Sep 03 2013 09:27:11 PM +00:00

For those interested in experimenting with $out in the development release (v2.5.2), this feature is documented in the release notes: http://docs.mongodb.org/manual/release-notes/2.6/#aggregation-pipeline-changes

Show

Dan Pasette added a comment - Sep 03 2013 09:27:11 PM +00:00 For those interested in experimenting with $out in the development release (v2.5.2), this feature is documented in the release notes: http://docs.mongodb.org/manual/release-notes/2.6/#aggregation-pipeline-changes

Hide

Permalink

auto (Inactive) added a comment - Oct 11 2013 10:38:54 PM +00:00

Author:

{u'username': u'dannenberg', u'name': u'Matt Dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: ~~SERVER-3253~~ expand $out testing
Branch: master
https://github.com/mongodb/mongo/commit/6ba8fc2f0ee77cffd2270556afc5984e44c1b243

Show

auto (Inactive) added a comment - Oct 11 2013 10:38:54 PM +00:00 Author: {u'username': u'dannenberg', u'name': u'Matt Dannenberg', u'email': u'matt.dannenberg@10gen.com'} Message: SERVER-3253 expand $out testing Branch: master https://github.com/mongodb/mongo/commit/6ba8fc2f0ee77cffd2270556afc5984e44c1b243

People

Assignee:

Mathias Stearn

Reporter:

Dan Pasette

Participants:

Alex Piggott, Andreas Petersson, Arthur Nogueira Neves, Asya Kamsky, auto, Bastien Barre, Bob Tiernay, BreakPhreak, Chris Westin, Colin Mollenhour, Dan Pasette, Jordan Willis, Keaton Adams, Ken Williams, Mathias Stearn, saks, SaurabhSanthosh, Shane R. Spencer, Shunsuke Mikami

Votes:

90 Vote for this issue

Watchers:

62 Start watching this issue

Dates

Created:

Jun 13 2011 07:08:48 PM +00:00

Updated:

May 17 2017 05:26:12 PM +00:00

Resolved:

Aug 08 2013 08:34:15 PM +00:00

Agile

View on Board