Uploaded image for project: 'Drivers'
  1. Drivers
  2. DRIVERS-2648

Add driver tests for Serverless Proxy incremental rollout

XMLWordPrintableJSON

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Component/s: Serverless
    • None
    • Not Needed
    • Hide

      Cloud is implementing a new serverless proxy that will be incrementally rolled out to new and existing clusters over the coming months. The team is leveraging pymongo's test suite to verify compliance with driver specs. In order to help ensure compatibility with all of our drivers - especially where they differ subtly and unintentionally from our official specs - we have agreed to add tests against the new serverless proxy to each driver's CI suite.  There are known failures that are being tracked in DRIVERS-2812.  The Serverless Proxy tests should not be run on PR commits until those are addressed.

      1. DRIVERS-2812
      1. DRIVERS-2812
      1. DRIVERS-2812
      1. DRIVERS-2812

       

       

      Each driver must continue running their serverless test suite against the existing serverless project (which will remain configured for the current serverless proxy). Each driver must run that same serverless test suite against the new serverless project which has been configured to use the new serverless proxy. Please let james.kovacs@mongodb.com (DBX) and siyuan.zhou@mongodb.com (Serverless) know of any reproducible failures against the new serverless proxy.

      Note for implementers: there is a new SERVERLESS_REGION variable that must be set to US_EAST_1.  It is recommended that drivers follow the pattern used in https://github.com/mongodb/mongo-python-driver/commit/41a131ea1c15ffa969a14ce1334ce19837dc226b, as described in the Serverless README.

      Show
      Cloud is implementing a new serverless proxy that will be incrementally rolled out to new and existing clusters over the coming months. The team is leveraging pymongo's test suite to verify compliance with driver specs. In order to help ensure compatibility with all of our drivers - especially where they differ subtly and unintentionally from our official specs - we have agreed to add tests against the new serverless proxy to each driver's CI suite.  There are known failures that are being tracked in DRIVERS-2812 .  The Serverless Proxy tests should not be run on PR commits until those are addressed. DRIVERS-2812 DRIVERS-2812 DRIVERS-2812 DRIVERS-2812     Each driver must continue running their serverless test suite against the existing serverless project (which will remain configured for the current serverless proxy). Each driver must run that same serverless test suite against the new serverless project which has been configured to use the new serverless proxy. Please let james.kovacs@mongodb.com (DBX) and siyuan.zhou@mongodb.com (Serverless) know of any reproducible failures against the new serverless proxy. Note for implementers: there is a new SERVERLESS_REGION variable that must be set to US_EAST_1.  It is recommended that drivers follow the pattern used in https://github.com/mongodb/mongo-python-driver/commit/41a131ea1c15ffa969a14ce1334ce19837dc226b, as described in the Serverless README.
    • $i18n.getText("admin.common.words.hide")
      Key Status/Resolution FixVersion
      CDRIVER-4770 Won't Do
      CXX-2782 Won't Do
      CSHARP-4836 Fixed 2.23.0
      GODRIVER-3040 Fixed 2.0.0
      JAVA-5234 Done
      NODE-5731 Fixed 6.4.0
      MOTOR-1207 Duplicate
      PYTHON-4031 Fixed 4.7
      PHPLIB-1303 Done
      RUBY-3347 Fixed 2.20.0
      RUST-1796 Fixed 2.8.0
      $i18n.getText("admin.common.words.show")
      #scriptField, #scriptField *{ border: 1px solid black; } #scriptField{ border-collapse: collapse; } #scriptField td { text-align: center; /* Center-align text in table cells */ } #scriptField td.key { text-align: left; /* Left-align text in the Key column */ } #scriptField a { text-decoration: none; /* Remove underlines from links */ border: none; /* Remove border from links */ } /* Add green background color to cells with FixVersion */ #scriptField td.hasFixVersion { background-color: #00FF00; /* Green color code */ } /* Center-align the first row headers */ #scriptField th { text-align: center; } Key Status/Resolution FixVersion CDRIVER-4770 Won't Do CXX-2782 Won't Do CSHARP-4836 Fixed 2.23.0 GODRIVER-3040 Fixed 2.0.0 JAVA-5234 Done NODE-5731 Fixed 6.4.0 MOTOR-1207 Duplicate PYTHON-4031 Fixed 4.7 PHPLIB-1303 Done RUBY-3347 Fixed 2.20.0 RUST-1796 Fixed 2.8.0

      Summary

      Atlas Serverless team plans to roll out serverless proxy incrementally in CLOUDP-145502. Serverless Proxy will be deployed between Cloud Load Balancers and Atlas Proxy, replacing the vanilla Envoy. Serverless Proxy will terminate client TLS connections and transcode requests to gRPC when forwarding them to the Atlas Proxy. Serverless Proxy also handles hello commands and process authentication commands specially.

      During the incremental rollout, we will only enable the new behavior for some tenants using SCRAM authentication, while keeping the current behavior for others, e.g. those that haven't be included in the rollout and those using X.509. Since Serverless Proxy essentially handles connection management and authentication for Serverless with dramatic changes like using gRPC, it's critical to test it's compatible with drivers.

      I'd propose to duplicate the driver serverless tests for Serverless Proxy's new behavior on 1-2 popular drivers, ideally node.js or python. As far as I know, we need to

      • add new test in the Evergreen config.
      • update create-instance.sh to run against the pre-defined serverless cluster in rollout or call Atlas test-only API to enable the rollout on Dev.
      • make sure only SCRAM is used in the tests, otherwise, disable other authentication methods.

      I wonder if the plan makes sense to Driver team and whether you have bandwidth later this quarter. Alternatively, Atlas Serverless are happy to add the tests with the Driver team's guidance and code review.

      Motivation

      Who is the affected end user?

      Atlas Serverless team.

      How does this affect the end user?

      This is to add correctness test coverage for an internal architectural change.

      Is this issue urgent?

      Atlas Serverless team plans to start testing in a month.

      Is this ticket required by a downstream team?

      Atlas Serverless

      Is this ticket only for tests?

      Only for tests.

      Acceptance Criteria

      Add driver tests that run against serverless clusters that are included in the Serverless Proxy incremental rollout.

            Assignee:
            steve.silvester@mongodb.com Steve Silvester
            Reporter:
            siyuan.zhou@mongodb.com Siyuan Zhou
            James Kovacs James Kovacs
            KeAna Moutra KeAna Moutra
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: