[DRIVERS-823] Support $merge and $out executing on secondaries Created: 04/Feb/20  Updated: 21/Aug/23

Status: Implementing
Project: Drivers
Component/s: CRUD
Fix Version/s: None

Type: Epic Priority: Major - P3
Reporter: Backlog - Core Eng Program Management Team Assignee: Jeremy Mikola
Resolution: Unresolved Votes: 0
Labels: big-rock
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by DOCS-13479 Investigate changes in SERVER-45974: ... Closed
Duplicate
is duplicated by DRIVERS-868 Support running $out or $merge from a... Closed
Issue split
split to PHPLIB-651 Support $merge and $out executing on ... Closed
split to PYTHON-2554 Support $merge and $out executing on ... Closed
split to RUBY-2526 Support $merge and $out executing on ... Closed
Related
related to DRIVERS-1969 Ignore read preference for $out/$merg... Closed
related to DRIVERS-1956 Skip $out CRUD tests on serverless en... Implementing
related to DRIVERS-1958 aggregate-write-readPreference tests ... Implementing
Driver Changes: Needed
Server Compat: 4.4
Quarter: FY22Q3
Upstream Changes Summary:

If an $out or $merge aggregate operation is delivered to a secondary node, and the operation has non-primary read preference, the operation will now work as expected. In previous versions, the operation would fail with a NotMaster category error. However, drivers have chosen not to add explicit support for this feature in their 4.4-compatible releases. There may be a way in some drivers to access this feature using a lower-level API. However, since the drivers are not changing to respect non-primary read preference for $out/$merge operations in 4.4, there should be no immediate downstream impact.

Downstream Changes Summary:

As of MongoDB 5.0, drivers can support read preferences (either explicit/per-operation or inherited) for aggregations including an $out/$merge stage. Previously, such pipelines would always require a primary.

Drivers should sync unified CRUD tests to mongodb/specifications@5f8f668. If possible, drivers may consider implementing DRIVERS-1958 alongside this ticket and sync to mongodb/specifications@dce2c48, which fixes an issue with the original spec tests for DRIVERS-823.

The outdated "$out Aggregation Pipeline Operator" specification was also deleted. All relevant information for $out/$merge is now discussed in the CRUD spec (and parts of Server Selection).

Note: DRIVERS-1969 revises behavior for applying a read preference. Drivers that have yet to implement DRIVERS-823 should consider that ticket when picking up this work. DRIVERS-1969 does not change any spec tests.

Start date:
Detailed Project Statuses:

Lead: Jeremy

Summary: Allow an aggregation with a $merge or $out stage to respect the user-specified read preference rather than unconditionally using read preference primary.

2021-09-22:

  • Spec PR close to getting wrapped up
  • Go and PHP implementations in code review

Driver Compliance:
Key Status/Resolution FixVersion
RUBY-2526 Fixed 2.17.0
PYTHON-2554 Fixed 4.1
RUST-663 Backlog
GODRIVER-1868 Done 1.8.0
JAVA-3994 Fixed 4.4.0
CXX-2184 Done 3.8.0
MOTOR-666 Duplicate
CDRIVER-3893 Done 1.21.0
SWIFT-1105 Won't Do
NODE-3083 Fixed 4.2.0
CSHARP-3397 Done 2.15.0
PHPLIB-651 Fixed 1.10.0

 Description   

Description of Linked Ticket

Epic Summary

Summary

Motivation

Cast of Characters

  • Product Owner: Katya Kamenieva
  • Project Lead: David Storch
  • Program Manager: Craig Homa.

Documentation

Server Scope Document
Server Technical Design Document
Driver Scope Document
Slack channel: #drivers-823-out-merge-on-secondaries



 Comments   
Comment by Jeremy Mikola [ 20/Oct/21 ]

DRIVERS-1958 made a correction to the spec tests for this issue. Although DRIVERS-1958 has its own language tickets, I've amended the downstream changes for DRIVERS-823 to refer to DRIVERS-1958 and suggest that drivers do both tickets together.

Comment by Githook User [ 28/Sep/21 ]

Author:

{'name': 'Jeremy Mikola', 'email': 'jmikola@gmail.com', 'username': 'jmikola'}

Message: DRIVERS-823: Support $out and $merge on secondaries (#1062)

  • Fix grammar and indentation for may-use-secondary commands
  • Delete obsolete $out aggregation pipeline operator spec
  • Fix RST warnings in CRUD spec
  • Suggest clarifying documentation for $merge output cursors
  • Clarify setting SecondaryOk/$readPreference for replica sets
  • Consolidate $out/$merge tests and add tests for db-aggregate
  • Drivers SHOULD use single selection attempt, but MAY use a second
  • mongos cannot combine db-aggregate stages with $out or $merge
  • Clarify concept of an effective read preference

Co-authored-by: Kevin Albertson <kevin.albertson@10gen.com>
Branch: master
https://github.com/mongodb/specifications/commit/5f8f6689538a39a36186285e747d3ac2fbcb27d1

Generated at Thu Feb 08 08:22:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.