[SERVER-29339] allow using $reduce expression as accumulator in $group Created: 23/May/17  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Asya Kamsky Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 7
Labels: accumulator, expression
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-29161 Ability to access previous document i... Closed
is related to SERVER-16284 Allow for $pushall / $push with $each... Open
is related to SERVER-4437 aggregation: support windowing opera... Closed
Assigned Teams:
Query Optimization
Participants:

 Description   

If $group accepted $reduce as accumulator, it would allow expressing any and all accumulator operations that we don't have implemented.

Proposed syntax:

{$group:{_id:<expr>,
      g:{$reduce:{
             input:<expr>,
             initialValue:<expr>,
             in:<expr> // using "$$value" and "$$this" same as the $reduce expression
      } }
} }



 Comments   
Comment by David Percy [ 10/Sep/20 ]

This would be great!

How would this interact with sharding though? Currently, $reduce guarantees an ordering (a left-fold), which matters if your accumulated $$value has a different shape than the input, $$this. But in a sharded $group we need to combine $$value1 and $$value2 from two different shards.

So maybe the accumulator version of $reduce shouldn't guarantee that ordering: $$this might be a new input, or a $$value from another shard. Maybe also we don't need to initialValue, because we never create empty groups.

> db.c.find()
{ _id: "a" }
{ _id: "b" }
{ _id: "c" }
{ _id: "d" }
> db.c.aggregate({$group: {_id: 1, v: {$reduce: {
    input: "$_id",
    in: {$concat: ["(", $$value, " ", $$this, ")"]},
  } } } })
{ _id: 1, v: "((a b) (c d))"}

Comment by Carlos Barilatti [ 05/Jun/19 ]

Would be a great functionality and it would also save a lo lot time.

Comment by Asya Kamsky [ 02/Jun/17 ]

This would allow accumulators to be created from any expression that already works on arrays.

Examples: $concat, $concatArrays.

Also ones that we don't have but have had requests for:
SERVER-15697
SERVER-4929
SERVER-16284
SERVER-7463
and even possibly SERVER-9377

Comment by Asya Kamsky [ 02/Jun/17 ]

The use cases for this are possibly related to SERVER-29161

The use case for being able to custom $reduce while accumulating may be doable with access to $$PREVIOUS, since $$PREVIOUS can now act as "$$value" does in $reduce.

The difference would be that having access to $$PREVIOUS would not expand your set of accumulator expressions in $group, so I think it wouldn't quite give you the flexibility $reduce would provide.

But having $reduce as accumulator would not make this ticket unnecessary, though there are some use cases that could be accomplished with either of these if both were implemented.

Generated at Thu Feb 08 04:20:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.