[CXX-2087] Implement All Relevant Spec Tests Created: 06/Aug/20  Updated: 31/Jan/24

Status: Backlog
Project: C++ Driver
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Epic Priority: Major - P3
Reporter: Alexander Golin (Inactive) Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by CXX-2108 Support parsing $uuid as extended JSO... Backlog
is depended on by CXX-2566 SDAM should prioritize electionId ove... Blocked
is depended on by CXX-2309 Test serialization of BSON with embed... Blocked
Related
related to CXX-2343 Load balancer: Add test tasks to ever... Backlog
related to CXX-2174 Define error handling behavior of wri... Closed
Quarter: FY25Q2

 Description   

Motivation

The C++ driver has historically chosen not to implement all spec tests, and instead rely on the C driver's existing testing. However, there is value in implementing all spec tests, even if there is a bit of redundancy.

  • It's not always clear which tests are implemented and why. In general the C++ driver relies on the C driver for much of the behavior, but there are exceptions (e.g. GridFS). By implementing all test runners, there'd be no question about which specification tests to run.
  • Swift has caught bugs / unexpected behavior in libmongoc when implementing spec tests (e.g. CDRIVER-3130CDRIVER-3147)
  • If the C++ driver does in the future remove the dependency of libbson/libmongoc, the testing will already be in place to enable such a transition.

Scope

Implement test runners for the following spec tests (specifications directory/file in parenthesis):

  • Auth tests (source/auth/tests)
  • BSON corpus tests (source/bson-corpus/tests)
  • Causal consistency prose tests (causal-consistency/causal-consistency.rst)
  • Connection string (source/connection-string)
  • Connections survive step down (source/connections-survive-step-down/tests)
  • Initial DNS seedlist discovery (source/initial-dns-seedlist-discovery/tests)
  • Max staleness (source/max-staleness/tests)
  • OCSP (source/ocsp-support/tests)
  • Retryable writes (source/retryable-writes/tests)
  • Server discovery and monitoring (source/server-discovery-and-monitoring/tests)
  • Server selection (source/server-selection/tests)
  • Sessions (source/sessions/tests)
  • URI options (source/uri-options/tests)
     
    It may be best to wait until the Unified Test Format project is complete, as that may simplify many of the existing test formats.


 Comments   
Comment by Kevin Albertson [ 09/Oct/23 ]

When starting this project, consider implementing only the tests where the C++ driver includes code separate from the C driver to fully implement the feature. Adding the test runners may add significant maintenance work to update tests and ensure tests pass in CI.

Testing features entirely handled in the C driver (example: OCSP) may not provide much value. Features have C++ driver API (example: Sessions) or integration code may provide value to test.

Generated at Wed Feb 07 22:04:51 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.