Uploaded image for project: 'Drivers'
  1. Drivers
  2. DRIVERS-2585

Use AWS Secrets Manager for AWS-Related Test Secrets

    XMLWordPrintableJSON

Details

    • Icon: Epic Epic
    • Resolution: Unresolved
    • Icon: Unknown Unknown
    • None
    • None
    • None
    • Needed
    • 0
    • 0
    • 0
    • 100
    • Hide

      Engineer(s): Noah Stapp
      Summary: Migrate AWS Secrets to AWS Secret Manager from Evergreen Project Variables.

      2023-09-15:

      • Status update:
        • Completed AWS tests with the Python driver.
        • Paused work to focus on other quarterly tasks.

      2023-09-01:

      • Status update:
        • Finished Atlas connection tests, wrapping up AWS tests with Python Driver.  Go Driver has implemented Atlas connection tests.  OIDC is also being migrated as part of DRIVERS-2415 updates this quarter.
      • Risks or delays:
        • Some secret values may need to be re-generated if the original source is lost
        • Variations in Evergreen project configuration have required additional work to generalize AWS Secret integrations.

      2023-08-21:

      • Status update:
        • First implementation in Python underway, steadily progressing through test suites.
        • Separating each test suite's secrets into separate vaults for better security. 
      • Risks or delays:
        • Some secret values may need to be re-generated if the original source is lost
        • Possible variations in Evergreen project configuration could require additional work to generalize AWS Secret integrations.
      Show
      Engineer(s): Noah Stapp Summary:  Migrate AWS Secrets to AWS Secret Manager from Evergreen Project Variables. 2023-09-15 : Status update: Completed AWS tests with the Python driver. Paused work to focus on other quarterly tasks. 2023-09-01 : Status update: Finished Atlas connection tests, wrapping up AWS tests with Python Driver.  Go Driver has implemented Atlas connection tests.  OIDC is also being migrated as part of DRIVERS-2415 updates this quarter. Risks or delays: Some secret values may need to be re-generated if the original source is lost Variations in Evergreen project configuration have required additional work to generalize AWS Secret integrations. 2023-08-21 : Status update: First implementation in Python underway, steadily progressing through test suites. Separating each test suite's secrets into separate vaults for better security.  Risks or delays: Some secret values may need to be re-generated if the original source is lost Possible variations in Evergreen project configuration could require additional work to generalize AWS Secret integrations.
    • $i18n.getText("admin.common.words.hide")
      Key Status/Resolution FixVersion
      CDRIVER-4701 Blocked
      CXX-2724 Blocked
      CSHARP-4741 Blocked
      GODRIVER-2928 Blocked
      JAVA-5094 Blocked
      NODE-5507 Blocked
      MOTOR-1167 Blocked
      PYTHON-3895 Blocked
      PHPLIB-1216 Blocked
      RUBY-3311 Blocked
      RUST-1717 Blocked
      $i18n.getText("admin.common.words.show")
      #scriptField, #scriptField *{ border: 1px solid black; } #scriptField{ border-collapse: collapse; } #scriptField td { text-align: center; /* Center-align text in table cells */ } #scriptField td.key { text-align: left; /* Left-align text in the Key column */ } #scriptField a { text-decoration: none; /* Remove underlines from links */ border: none; /* Remove border from links */ } /* Add green background color to cells with FixVersion */ #scriptField td.hasFixVersion { background-color: #00FF00; /* Green color code */ } /* Center-align the first row headers */ #scriptField th { text-align: center; } Key Status/Resolution FixVersion CDRIVER-4701 Blocked CXX-2724 Blocked CSHARP-4741 Blocked GODRIVER-2928 Blocked JAVA-5094 Blocked NODE-5507 Blocked MOTOR-1167 Blocked PYTHON-3895 Blocked PHPLIB-1216 Blocked RUBY-3311 Blocked RUST-1717 Blocked

    Description

      Summary

      _We currently have around 20 Evergreen Project variables that are used to populate a ${DRIVERS_TOOLS}/.evergreen/auth_aws/aws_e2e_setup.json file that is used in Drivers Evergeen Tools in test scripts. As part of DRIVERS-2415, we now have a mechanism to store and retrieve variables using AWS Secrets Manager, rather than continuing to grow this list of manually updated variables across all drivers.

      All members of dbx have access to view and update the secrets using the drivers-test-secrets-role login option in the Drivers AWS account.

      This project would move the existing affected Project Variables and create a new wiki page for the maintenance and upkeep of these secrets. There would be a new script created in Drivers Evergreen Tools to create an expansion file used by EG to provide these values as environment variables which can then be used by the existing scripts instead of loading the values from aws_e2e_setup.json.

      Drivers would then replace the portion of their Evergreen Config with a block that acquires the appropriate credentials and expands the variables. They would also be able to remove the affected project variables from EG.

      Motivation

      Adding and updating credentials currently requires the coordination of all of the driver teams, and manual effort.

      Attachments

        Activity

          People

            noah.stapp@mongodb.com Noah Stapp
            steve.silvester@mongodb.com Steve Silvester
            Steven Silvester Steven Silvester
            Esha Bhargava Esha Bhargava
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: