[DRIVERS-776]  Support 'let' option for aggregate command Created: 02/Dec/19  Updated: 27/May/22  Resolved: 01/Mar/22

Status: Closed
Project: Drivers
Component/s: CRUD
Fix Version/s: None

Type: Spec Change Priority: Major - P3
Reporter: Backlog - Core Eng Program Management Team Assignee: Jeremy Mikola
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-57403 Serialization error in let variables ... Closed
depends on SERVER-46707 Support 'let' variables in aggregate ... Closed
depends on SERVER-46115 Implement 'let' parameters to aggrega... Closed
depends on SERVER-46998 Support 'let' variables used within $... Closed
Issue split
split to JAVA-4186 Support 'let' option for aggregate c... Closed
split to PHPLIB-663 Support 'let' option for aggregate c... Closed
split to RUST-836 Support 'let' option for aggregate c... Closed
split to CDRIVER-4010 Support 'let' option for aggregate c... Closed
split to CSHARP-3700 Support 'let' option for aggregate c... Closed
split to CXX-2274 Support 'let' option for aggregate c... Closed
split to GODRIVER-2033 Support 'let' option for aggregate c... Closed
split to MOTOR-745 Support 'let' option for aggregate c... Closed
split to NODE-3325 Support 'let' option for aggregate c... Closed
split to PYTHON-2741 Support 'let' option for aggregate c... Closed
split to RUBY-2659 Support 'let' option for aggregate c... Closed
split to NODE-3353 Test for aggregate $out stage using t... Closed
Related
Driver Changes: Needed
Server Compat: 4.7, 5.0, 5.1
Quarter: FY22Q2
Upstream Changes Summary:

Added let parameters to the aggregate command.

Driver Compliance:
Key Status/Resolution FixVersion
CDRIVER-4010 Fixed 1.18.0
CXX-2274 Fixed 3.7.0
CSHARP-3700 Fixed 2.13.0
GODRIVER-2033 Fixed 1.6.0
JAVA-4186 Fixed 4.3.0
NODE-3325 Fixed 4.0.0
MOTOR-745 Fixed 2.5
PYTHON-2741 Fixed 3.12, 4.0
PHPLIB-663 Fixed 1.9.0
RUBY-2659 Fixed 2.17.0
RUST-836 Fixed 2.0.0-beta.3
SWIFT-1225 Fixed 1.2.0
NODE-3353 Fixed 4.1.0

 Description   
Downstream Change Summary

Drivers should add a new let option to aggregate helper methods and sync CRUD unified spec tests with mongodb/specifications@3a58fc6.

For MongoDB 5.0+, the aggregate command accepts a let option of type object. This object consists of any amount of parameter names each followed by definitions of constants in the MQL Aggregate Expression language. Each parameter name is then usable to access the value of the corresponding MQL Expression with the "$$" syntax within Aggregate Expression contexts which may require the use of $expr or a pipeline.

Note: this ticket only pertains to changes for aggregate. Support for let in other commands (involving pipelines or $expr) will be handled by DRIVERS-1788.

Description of Linked Ticket

Epic Summary

Summary

Deprecate the MR command and ensure its functionality is fully supported by Agg. The backend of the command has already been replaced by “MapReduce in Agg” (PM-776), we need to make sure all necessary concepts can be expressed in MQL.

Motivation

One objective of the query team is to consolidate dialects of all existing query commands into a single language. This project enables users to write their MR tasks directly in Agg, which provides a more capable and clear syntax. We want to move users away from legacy ways of interacting with the system. Independently, from MR jobs, some extensions are valuable for the users.

Documentation

Scope Document
Technical Design Document



 Comments   
Comment by Jeremy Mikola [ 10/Jun/21 ]

Note: description was updated to point to mongodb/specifications@3a58fc6 for syncing tests in order to incorporate the latest test changes from mongodb/specifications#1014.

Comment by Githook User [ 10/Jun/21 ]

Author:

{'name': 'Jeff Yemin', 'email': 'jeff.yemin@mongodb.com', 'username': 'jyemin'}

Message: DRIVERS-776: Add tests for aggregate with $out using let option (#1014)
Branch: master
https://github.com/mongodb/specifications/commit/3a58fc612b3520030a97a1161a5651c451c40f8e

Comment by Githook User [ 07/Jun/21 ]

Author:

{'name': 'Jeremy Mikola', 'email': 'jmikola@gmail.com', 'username': 'jmikola'}

Message: DRIVERS-776: Split tests to work around SERVER-57403 (#1013)
Branch: master
https://github.com/mongodb/specifications/commit/9465cda31270e41d3d87fe9b9b14ad601c3861c6

Comment by Jeremy Mikola [ 07/Jun/21 ]

New PR to split the original test to work around a mongos bug (SERVER-57403):

https://github.com/mongodb/specifications/pull/1013

Comment by Githook User [ 02/Jun/21 ]

Author:

{'name': 'Jeremy Mikola', 'email': 'jmikola@gmail.com', 'username': 'jmikola'}

Message: DRIVERS-776: Add let to AggregateOptions (#1008)

Also fixes RST backtick formatting in various places
Branch: master
https://github.com/mongodb/specifications/commit/3db387995bc0e64df862d621aaa31387afc1537c

Comment by Jeremy Mikola [ 01/Jun/21 ]

https://github.com/mongodb/specifications/pull/1008

Comment by James Wahlin [ 01/Jun/21 ]

It would be fine to prioritize support for 'let' parameters for the aggregate command first. This feature is needed for some users to rewrite mapReduce commands to aggregates. We plan to deprecate mapReduce for 5.0.

Adding 'let' parameter support for the other commands can trail and can be done post 5.0.

cc: rachelle.palmer kateryna.kamenieva pawel.terlecki jacob.evans

Comment by Jeremy Mikola [ 28/May/21 ]

alexander.golin just created DRIVERS-1788 to handle downstream changes for adding let to commands other than aggregate. This issue has been modified to encompass all changes to aggregate and we're including additional tickets for support on sharded clusters and $merge pipelines. This way, drivers can simply add the option and not have to worry about raising some client-side error if it's unsupported (e.g. detecting $merge somewhere in the pipeline). Additionally, our CRUD spec tests will be able to target any cluster.

I assume there's no concern for serverless deployments, and those can support let just fine.

cc: pawel.terlecki, james.wahlin

Comment by Jeremy Mikola [ 28/May/21 ]

Just copied in the downstream changes from PM-1470. I assume that might have been entered after this ticket was spawned, which might explain why the downstream changes in this description was "null".

Comment by Anna Henningsen [ 28/May/21 ]

Concretely, the let option (as documented on https://docs.mongodb.com/v5.0/reference/command/aggregate/#mongodb-dbcommand-dbcmd.aggregate) needs to be allowed in AggregateOptions and actually passed through to the aggregate command when .aggregate() is called, and the same for several other commands (see the now-up-to-date downstream changes summary here).

I assume most of the work here would be writing tests for this.

Comment by Kaitlin Mahar [ 28/May/21 ]

Putting back into "needs triage" as anna.henningsen notes there are changes required for at least one driver, and to AggregateOptions in the CRUD spec.

Comment by Esha Bhargava [ 09/Jul/20 ]

No Drivers changes needed.

Generated at Thu Feb 08 08:22:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.