[DRIVERS-2322] Restrict change streams unified tests to replicasets Created: 10/May/22  Updated: 31/May/22

Status: Implementing
Project: Drivers
Component/s: Change Streams
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Benji Rewis (Inactive) Assignee: Benji Rewis (Inactive)
Resolution: Unresolved Votes: 0
Labels: yaml-test
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by GODRIVER-2362 Change Stream event document missing ... Closed
is depended on by DRIVERS-2323 Run all possible change streams spec ... Backlog
is depended on by GODRIVER-1986 Convert change stream spec tests to u... Closed
is depended on by GODRIVER-2296 Do not error when parsing change stre... Closed
Issue split
split to CDRIVER-4391 Restrict change streams unified tests... Backlog
split to CSHARP-4176 Restrict change streams unified tests... Closed
split to CXX-2514 Restrict change streams unified tests... Closed
split to GODRIVER-2422 Restrict change streams unified tests... Closed
split to MOTOR-964 Restrict change streams unified tests... Closed
split to NODE-4252 Restrict change streams unified tests... Closed
split to PHPLIB-873 Restrict change streams unified tests... Closed
split to PYTHON-3273 Restrict change streams unified tests... Closed
split to RUBY-3001 Restrict change streams unified tests... Closed
split to RUST-1327 Restrict change streams unified tests... Closed
split to JAVA-4617 Restrict change streams unified tests... Closed
Related
related to PHPLIB-842 Drop operation from initialData unexp... Backlog
Driver Changes: Needed
Downstream Changes Summary:

Drivers should sync their unified/change-streams.[yml/json] tests to this commit. We've restricted all tests in that file to only run against replicasets for now (many of them already have this restriction) until we have time to look into sharded cluster failures with DRIVERS-2323. If any drivers are not seeing sharded cluster failures, this restriction and spec test update is optional.

Driver Compliance:
Key Status/Resolution FixVersion
CDRIVER-4391 Backlog
CXX-2514 Fixed 3.7.0
CSHARP-4176 Fixed 2.16.0
GODRIVER-2422 Done
JAVA-4617 Fixed 4.7.0
NODE-4252 Done 4.7.0
MOTOR-964 Duplicate
PYTHON-3273 Won't Do
PHPLIB-873 Fixed 1.13.0-beta1, 1.13.0
RUBY-3001 Fixed 2.18.0
RUST-1327 Fixed 2.3.0
SWIFT-1570 Duplicate

 Description   

Summary

A number of the change streams spec tests in change-streams.yml already only run against replicasets (standalones do not support change streams). Running against regular sharded clusters and sharded replica sets causes unexpected behavior in the command monitoring events observed (usually unexpected drops and writes that cause test failures). The cause of these unexpected events is not immediately clear, so we should restrict the tests in that file to only run against replicasets for the time being. We can re-enable testing against sharded clusters as part of DRIVERS-2323.

Motivation

Who is the affected end user?

Driver devs.

How does this affect the end user?

A couple teams (at least Go and PHP) are blocked on merging change streams tests because of failures against sharded clusters that likely do not represent bugs in the drivers.

How likely is it that this problem or use case will occur?

Consistent failures against sharded clusters.

If the problem does occur, what are the consequences and how severe are they?

This is blocking at least a couple tickets in the Go driver.

Is this issue urgent?

Somewhat.

Is this ticket required by a downstream team?

No.

Is this ticket only for tests?

Yes.



 Comments   
Comment by Benji Rewis (Inactive) [ 31/May/22 ]

steve.silvester@mongodb.com made a comment in the downstream changes summary to allow drivers to skip this test sync if they are not seeing sharded cluster failures. Interesting that Python has none; that may be a clue to the eventual resolution of DRIVERS-2323.

Comment by Steve Silvester [ 24/May/22 ]

We're running the tests with sharded replica sets against server versions 3.6+ in https://github.com/mongodb/mongo-python-driver/pull/950, so we are not planning to implement this ticket in Python. Perhaps this ticket should be modified to suggest restricting the tests only if there are actual test failures?

Comment by Githook User [ 17/May/22 ]

Author:

{'name': 'Benjamin Rewis', 'email': '32186188+benjirewis@users.noreply.github.com', 'username': 'benjirewis'}

Message: DRIVERS-2322 Restrict most unified change streams tests to only run against replicasets (#1210)

Co-authored-by: Jeremy Mikola <jmikola@gmail.com>
Branch: master
https://github.com/mongodb/specifications/commit/f07351c0c61cd79e0e7a197f78a5cfcc028c2966

Comment by Benji Rewis (Inactive) [ 16/May/22 ]

https://github.com/mongodb/specifications/pull/1210

Generated at Thu Feb 08 08:25:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.