[DRIVERS-2799] Test-level runOnRequirements in retryable writes tests should not be more permissive than top-level runOnRequirements Created: 19/Dec/23  Updated: 20/Dec/23

Status: Implementing
Project: Drivers
Component/s: Retryability
Fix Version/s: None

Type: Task Priority: Minor - P4
Reporter: Jeremy Mikola Assignee: Jeremy Mikola
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Issue split
split to CSHARP-4887 Test-level runOnRequirements in retry... Backlog
split to CXX-2798 Test-level runOnRequirements in retry... Backlog
split to GODRIVER-3073 Test-level runOnRequirements in retry... Backlog
split to JAVA-5272 Test-level runOnRequirements in retry... Backlog
split to PYTHON-4110 Test-level runOnRequirements in retry... Backlog
split to RUST-1811 Test-level runOnRequirements in retry... Backlog
split to CDRIVER-4798 Test-level runOnRequirements in retry... Closed
split to MOTOR-1226 Test-level runOnRequirements in retry... Closed
split to PHPLIB-1326 Test-level runOnRequirements in retry... Closed
split to RUBY-3361 Test-level runOnRequirements in retry... Closed
split to NODE-5797 Test-level runOnRequirements in retry... Scheduled
Related
is related to DRIVERS-1385 Make retryable write test error label... Implementing
Driver Changes: Needed
Downstream Changes Summary:

Summary of necessary driver changes

  • No driver changes needed beyond syncing spec testsĀ 

Commits for syncing spec/prose tests

  • Sync unified spec tests for retryable-writes and unified-test-format/valid-pass with 4c73bb1
Engineering Lead: Andreas Braun Andreas Braun
Start date:
Driver Compliance:
Key Status/Resolution FixVersion
CDRIVER-4798 Fixed 1.26.0
CXX-2798 Backlog
CSHARP-4887 Backlog
GODRIVER-3073 Backlog 2.1.0
JAVA-5272 Backlog
NODE-5797 Scheduled
MOTOR-1226 Duplicate
PYTHON-4110 Backlog 4.7
PHPLIB-1326 Fixed 1.18.0
RUBY-3361 Fixed 2.20.0
RUST-1811 Backlog

 Description   

Summary

The unified spec tests introduced in ab48c57 for DRIVERS-1385 incorrectly specify top-level runOnRequirements that are more restrictive than the test-level runOnRequirements.

Those tests may have been copied from the original poc-retryable-writes.yml tests introduced in e4808b8 for DRIVERS-709, which have the same issue.

This is contrary to the advice in the Unified Test Format spec:

[Test-level] requirements SHOULD be more restrictive than those specified in the top-level runOnRequirements (if any) and SHOULD NOT be more permissive. This is advised because both sets of requirements MUST be satisified in order for a test to be executed and more permissive requirements at the test-level could be taken out of context on their own.

As currently written, the spec tests will never run on sharded clusters and load balancers because the top-level runOnRequirements require a replica set.

Motivation

Is this issue urgent?

No.

Is this ticket required by a downstream team?

No.

Is this ticket only for tests?

Yes.



 Comments   
Comment by Githook User [ 20/Dec/23 ]

Author:

{'name': 'Jeremy Mikola', 'email': 'jmikola@gmail.com', 'username': 'jmikola'}

Message: DRIVERS-2799: Use more permissive top-level runOnRequirements (#1484)

  • DRIVERS-2799: Use more permissive top-level runOnRequirements

This avoids unintentionally skipping tests. Previously, some tests were never run on sharded clusters.

  • Add RetryableWriteError error label via failCommand fail point

This ensures the unified test is consistent with the legacy test from which it was derived. Additionally, it makes the test resilient to possible changes in DRIVERS-1641.

This inconsistency dates back to e4808b84f804f8b08293dcd51ce107e10a40589f.
Branch: master
https://github.com/mongodb/specifications/commit/4c73bb13daa0241a3956a30b451edda10f79dde6

Comment by Jeremy Mikola [ 19/Dec/23 ]

https://github.com/mongodb/specifications/pull/1484

Comment by Jeremy Mikola [ 19/Dec/23 ]

Encountered this while working on DRIVERS-1641.

Generated at Thu Feb 08 08:26:26 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.