[DRIVERS-1375] Run Test Suites Against Serverless in Evergreen Created: 18/Aug/20  Updated: 21/Aug/23

Status: Implementing
Project: Drivers
Component/s: None
Fix Version/s: None

Type: Epic Priority: Major - P3
Reporter: Louisa Berger Assignee: Kevin Albertson
Resolution: Unresolved Votes: 0
Labels: serverless-public-beta
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
is duplicated by PYTHON-2125 Use the Python driver test suite to t... Closed
Initiative
Issue split
split to CDRIVER-3886 Run Test Suites Against Serverless in... Backlog
split to MOTOR-658 Run Test Suites Against Serverless in... Closed
split to PHPLIB-683 Run Test Suites Against Serverless in... Closed
split to RUBY-2518 Run Test Suites Against Serverless in... Closed
Related
is related to PYTHON-2125 Use the Python driver test suite to t... Closed
Driver Changes: Needed
Quarter: FY22Q2
Start date:
Cost Threshold %: 100
Detailed Project Statuses:

Summary
As part of the Serverless initiative, the Atlas Proxy will start presenting as a mongos to clients, while still being backed by a replica set. To make sure that this works as expected from the Drivers perspective, we should start running the Drivers tests against the "Proxy as a Mongos" to make sure that the Proxy behaves as expected.

Author & Lead Kevin Albertson

2021-07-16

  • Teams are currently blocked on CLOUDP-95406

2021-06-30

  • Swift and Ruby are complete
  • C, C#, Go, and Python and Java have this work in progress.
  • PHP is currently facing a cloud-side blocker and Andreas is corresponding in the serverless channel to get it addressed.
  • Node has this work scheduled for after their critical 5.0 work and their V4 breaking changes.
  • Rust will complete following 2.0 and performance work.
  • CXX will follow C due to staffing shortage.

2021-06-16

  • Teams are progressing - C, C#, Go, Ruby, and Swift are in progress
  • Java is blocked on BUILD-13305
  • PHP, Python, Node, C
  • Rust will complete following 2.0 and performance work.

2021-05-17

  • CLOUDP-88087 is complete and we now have the deployment of the proxy containing the fail point support we required. Teams are now able to complete this work as they have planned in QP. Not showing any risks for 5.0 GA.

2021-05-04

  • Cloud has implemented the support we require in the proxy, but we are now waiting on their deployment of these changes tracked in CLOUDP-88087

2021-04-06

  • Cloud has begun implementing support for failCommand in the proxy and meanwhile Patrick is trying to continue making headway on the DBX side while also juggling some additional Rust responsibilities due to Sam departing.
  • DBX teams have been making private beta compatibility versions and posting them to this wiki page: https://wiki.corp.mongodb.com/x/e5DxBw
  • We will plan to highly prioritize this proxy testing work as soon as we are unblocked, and meanwhile we may be able to make incremental headway running only CRUD tests.

2021-03-23

  • In the two weeks since the scope wrapped we ran into a roadblock, which is that drivers tests use failpoints and these would not work in the shared MT proxy.
  • We explored the option of testing in a local instance instead, however this approach would mean higher maintenance load because drivers would need to continually keep their local instance up to date.
  • We synced up with cloud folks and determined that it would be worth supporting failCommand in the proxy, but first we needed to ensure that future load balancer work wouldn't invalidate the use of failpoints in drivers tests. Conversations are ongoing in the load balancer and proxy testing project channels, however it appears that we are going to proceed with the plan of implementing failCommand support in the proxy via CLOUDP-85679.
  • We have decided that this project WILL NOT complete by the private beta target of March 31, and it WILL NOT block the private beta announcement. Timeline estimates TBD based on analysis of the work required in CLOUDP-85679.

2021-03-09
The scope has just wrapped up final review. Kevin is going on PTO for 2 weeks so has handed off the remaining design-like work to Patrick Freed, who is currently directing his focus here. Aiming to wrap up the remaining design-side work this week to unblock drivers to implement ahead of serverless private beta.

Driver Compliance:
Key Status/Resolution FixVersion
MOTOR-658 Duplicate
RUBY-2518 Done 2.15.0
CXX-2176 Backlog
PHPLIB-683 Fixed 1.12.0
RUST-656 Fixed 2.1.0
NODE-3072 Fixed 4.1.1
CSHARP-3387 Fixed 2.14.0
PYTHON-2545 Fixed 3.12.1, 4.0
JAVA-3987 Fixed 4.4.0
GODRIVER-1861 Fixed 1.8.0
CDRIVER-3886 Backlog
SWIFT-1143 Fixed 1.2.0

 Description   

Run a subset of specification tests against Serverless.



 Comments   
Comment by Githook User [ 18/Jun/21 ]

Author:

{'name': 'Dmitry Rybakov', 'email': 'dmitry.rybakov@mongodb.com', 'username': 'comandeo'}

Message: DRIVERS-1375 Make serverless scripts executable (#155)
Branch: master
https://github.com/mongodb-labs/drivers-evergreen-tools/commit/8830d07418523f75dc0a6593c4054a324f45988b

Comment by Matt Broadstone [ 24/Aug/20 ]

Linked to the "Automated Spec Testing" project, which could provide test coverage across all drivers for any YAML test written using the "Unified Test Spec" format. 

Generated at Thu Feb 08 08:23:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.