[SERVER-10626] Support numeric arrays as input to $add in aggregation's $project Created: 26/Aug/13 Updated: 30/May/19 Resolved: 23/Jul/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework |
| Affects Version/s: | 2.4.6 |
| Fix Version/s: | None |
| Type: | New Feature | Priority: | Major - P3 |
| Reporter: | Oleg Rekutin | Assignee: | James Cohan |
| Resolution: | Duplicate | Votes: | 2 |
| Labels: | pull-request | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Sprint: | Quint Iteration 7 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
The $add expression (as used in $project aggregation operator) supports the ability to have an explicit array of numbers to sum be specified, such as:
However, when same array numeric value is provided as an input, the addition/summation does not occur:
This summation is entirely contained within the document. Today, doing this requires an $unwind followed by a $group. However {$group: {_id: "$_id"}} does not stream and can easily hit the 16MB limit on large collections. Furthermore, it's a shame to use $unwind/{$group: {_id: "$_id"}}, as it's just a verbose projection. The proposal is to make the above the test case pass. This would need documentation updates. |
| Comments |
| Comment by Ian Whalen (Inactive) [ 26/Apr/19 ] | |||
|
Switching "Drivers Changes Needed" from "Maybe" to "Not Needed" since this was closed as something other than Fixed. | |||
| Comment by Githook User [ 11/Jun/15 ] | |||
|
Author: {u'username': u'cswanson310', u'name': u'Charlie Swanson', u'email': u'charlie.swanson@mongodb.com'}Message: Prep for | |||
| Comment by Oleg Rekutin [ 30/Jan/15 ] | |||
|
Asya, I agree, I don't see syntactic ambiguity in $add: "$arrayOfInts". I'll try to get a new pull request in against the latest code base. Andy, do you still think there is syntactic ambiguity here? It's been over a year since I filed this, and so it's possible that having worked a lot more in the aggregation pipeline since then will have given you new insight. | |||
| Comment by Asya Kamsky [ 17/Apr/14 ] | |||
|
I would like to be able to do $add:"$arrayOfInts" (though $sum:"$arrayOfInts" would work also, but $add is available in $project). I guess I'm confused how $add:[1,2,3,4] could be ambiguous language-wise. | |||
| Comment by Oleg Rekutin [ 29/Aug/13 ] | |||
|
Would it be better to change my patch to add $sum support to avoid the syntactic ambiguity and rule out vector addition? Allowing existing reduction operations, such as $sum, would be ideal. I would love to be able to $sum values (this JIRA), count array sizes ( Of course, this now starting to sound like, SERVER-4438, which is nested pipelines on arrays. But that sounds big and scary, as I haven't sat down to think about how that would look syntactically. It might be way overkill and difficult to use. What's hurting us right now is the inability to do any kinds of reductions on arrays within documents, without resorting to $unwind/$group({_id:"$_id"}). However, when you do that, you can very quickly hit the 16MB limit, making whole-collection aggregations infeasible. (Not to mention the sheer waste of unwinding potentially long arrays just to count the number of items or add the values). I strongly urge some kind of ability to do these reductions. This is severely limiting the power of the aggregation framework for documents in any collections that we know will keep growing (which, for us, is all of our collections). Aggregation becomes a toy feature after a few encounters with this limitation (which is unfair, given how fast and useful it can often be). | |||
| Comment by Andy Schwerin [ 29/Aug/13 ] | |||
|
Reducing values in arrays within documents is a concept that the projection language of aggregation intentionally omits, for the time being. The reason is that it introduces a lot of potential syntactic ambiguity (is it vector add or reduction, e.g.), and we're still working to resolve that. The underlying goal here is good, but I'm not convinced this is the way. An alternative to this that I've been idly considering is allowing the existing reduction operators ($sum in this case) to be used in projections to do array reduction. | |||
| Comment by Oleg Rekutin [ 26/Aug/13 ] | |||
|
The motivating use-case for this change is the following:
Here, a normal "$a.b" projection would've provided a $total array with numbers. With $add, the total can be obtained without unwind/group. | |||
| Comment by Oleg Rekutin [ 26/Aug/13 ] | |||
|
https://github.com/mongodb/mongo/pull/476 pull request Looking at v2.6, it's somewhat difficult to tell, but this might be addressed with $let? Either way, I think this would still be valuable in 2.6 due to simplicity of syntax. |