[SERVER-3253] aggregation: unsharded support $out Created: 13/Jun/11 Updated: 16/Nov/21 Resolved: 08/Aug/13 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework |
| Affects Version/s: | None |
| Fix Version/s: | 2.5.2 |
| Type: | New Feature | Priority: | Major - P3 |
| Reporter: | Daniel Pasette (Inactive) | Assignee: | Mathias Stearn |
| Resolution: | Done | Votes: | 89 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
Implement output option for aggregation pipelines |
| Comments |
| Comment by auto [ 11/Oct/13 ] |
|
Author: {u'username': u'dannenberg', u'name': u'Matt Dannenberg', u'email': u'matt.dannenberg@10gen.com'}Message: |
| Comment by Daniel Pasette (Inactive) [ 03/Sep/13 ] |
|
For those interested in experimenting with $out in the development release (v2.5.2), this feature is documented in the release notes: http://docs.mongodb.org/manual/release-notes/2.6/#aggregation-pipeline-changes |
| Comment by Shane R. Spencer [ 26/Aug/13 ] |
|
I believe using a temporary collection and renaming it should be optional. I would love to use capped or TTL collections for aggregation results. I'm not sure if making this optional would also allow sharding to be done. I'm not too familiar with where code is processed when mongos is involved. |
| Comment by auto [ 19/Aug/13 ] |
|
Author: {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}Message: Use bulk-insert in $out About 4x faster at simple copy-collection benchmark. |
| Comment by auto [ 26/Jul/13 ] |
|
Author: {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}Message: |
| Comment by Asya Kamsky [ 01/Jul/13 ] |
|
breakphreak I suggest you ask about this on google-groups (mongodb-user) because something seems very strange if find takes minutes for query similar to what you are aggregating in seconds. |
| Comment by BreakPhreak [ 30/Jun/13 ] |
|
+1 |
| Comment by Aliaksandr Rahalevich [ 26/Jun/13 ] |
|
+1 for this feature. Will appreciate a lot. |
| Comment by Ken Williams [ 24/Jun/13 ] |
|
+1. Count me as another vote for making this high-priority. Would be extremely helpful. |
| Comment by Bob Tiernay [ 15/May/13 ] |
|
This is crucial for any sizeable collection. Otherwise it may require reissuing the aggregation pipeline multiple times in order to page through the results. This is terribly inefficient. Is there an ETA on this feature? +2 |
| Comment by Keaton Adams [ 07/May/13 ] |
|
So in our current SQL solution, we do a number of INSERT INTO <Summary>, SELECT <ID Column>, SUM(VAL1), SUM(VAL2) WHERE <Created_TS> >= 10 min ago; Instead of a stored proc I would like to do this in a json script called via cron, but it sounds like I really need this $out function to keep things simple with directing the output from aggregate to another collection within the DB. If I understand this $out function properly, I really need this to mimic / duplicate what I can currently do in SQL. Thanks for considering it for the next release. |
| Comment by Jordan Willis [ 06/May/13 ] |
|
This would be huge for me and would really get the bioinformatics community switched over to mongodb for good. -J |
| Comment by Bastien Barre [ 15/Mar/13 ] |
|
really important. We need it to play with a lot of data. |
| Comment by Arthur Nogueira Neves [ 14/Mar/13 ] |
|
+1 .. really important one indeed |
| Comment by SaurabhSanthosh [ 10/Dec/12 ] |
|
This is a very important feature which makes the aggregation framework really useful(especially when we are dealing with huge data sets). +1 for this one. |
| Comment by Shunsuke Mikami [ 15/Oct/12 ] |
|
I agree with Andreas Petersson. |
| Comment by Andreas Petersson [ 05/Oct/12 ] |
|
+1 for this one. This is huge and I cant believe this is not voted higher. |
| Comment by Alex Piggott [ 14/Aug/12 ] |
|
Any chance of being able to use the output of an aggregation to update existing collections in addition to merging/replacing? I'm looking for a method of writing aggregations and other transforms back into existing collections without hitting the client or blocking the server - the "recommended" way of doing this is an unplanned improvement to update (https://jira.mongodb.org/browse/SERVER-458) that's been sat on the queue forever, currently I abuse a bug in map/reduce to perform the required functions (https://groups.google.com/forum/?fromgroups#!topic/mongodb-user/AxXY7r6hHc8%5B1-25%5D). Edit: I just noticed that a discussion about updating collections from the new aggregation framework is discussed in that link above: https://jira.mongodb.org/browse/SERVER-458 - semi-apologies for duplicating! |
| Comment by Chris Westin [ 15/Feb/12 ] |
|
That's not decided yet. The goal is to make it as similar to M/R as it makes sense to do so. Some modes may not make sense. |
| Comment by Colin Mollenhour [ 14/Feb/12 ] |
|
Will this have both replace and merge modes like M/R? |
| Comment by Chris Westin [ 16/Sep/11 ] |
|
|