Uploaded image for project: 'Drivers'
  1. Drivers
  2. DRIVERS-2657

Make Lambda Tests More Resilient

XMLWordPrintableJSON

    • Type: Icon: Spec Change Spec Change
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Component/s: FaaS
    • Labels:
      None
    • Needed
    • Hide

      Drivers that currently automate FaaS testing should update their Evergreen configurations to use task groups instead of the single tools script as described in https://github.com/mongodb/specifications/commit/8debfaebfc20402d26d8058661d51309ba5e2344 .

      Show
      Drivers that currently automate FaaS testing should update their Evergreen configurations to use task groups instead of the single tools script as described in https://github.com/mongodb/specifications/commit/8debfaebfc20402d26d8058661d51309ba5e2344 .
    • $i18n.getText("admin.common.words.hide")
      Key Status/Resolution FixVersion
      CDRIVER-4670 Won't Do
      CXX-2704 Won't Do
      CSHARP-4692 Duplicate
      GODRIVER-2880 Duplicate
      JAVA-5042 Backlog
      NODE-5373 Duplicate
      MOTOR-1147 Duplicate
      PYTHON-3775 Duplicate
      PHPLIB-1171 Won't Do
      RUBY-3283 Done
      RUST-1691 Duplicate
      $i18n.getText("admin.common.words.show")
      #scriptField, #scriptField *{ border: 1px solid black; } #scriptField{ border-collapse: collapse; } #scriptField td { text-align: center; /* Center-align text in table cells */ } #scriptField td.key { text-align: left; /* Left-align text in the Key column */ } #scriptField a { text-decoration: none; /* Remove underlines from links */ border: none; /* Remove border from links */ } /* Add green background color to cells with FixVersion */ #scriptField td.hasFixVersion { background-color: #00FF00; /* Green color code */ } /* Center-align the first row headers */ #scriptField th { text-align: center; } Key Status/Resolution FixVersion CDRIVER-4670 Won't Do CXX-2704 Won't Do CSHARP-4692 Duplicate GODRIVER-2880 Duplicate JAVA-5042 Backlog NODE-5373 Duplicate MOTOR-1147 Duplicate PYTHON-3775 Duplicate PHPLIB-1171 Won't Do RUBY-3283 Done RUST-1691 Duplicate

      Summary

      When the automated AWS Lambda tests error or are aborted, Atlas clusters run the risk of being orphaned. The current script attempts to handle this in a trap, but that is not always 100% effective.

      The recommendation is to split the script (https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/run-deployed-lambda-aws-tests.sh) into separate scripts that can be run in driver CI as task groups where the setup and teardown can be guaranteed regardless of error.

      This should/could also be given additional protection by using cloud-dev instead of cloud and setting auto-expiration times much lower that the default, if possible.

      Motivation

      Who is the affected end user?

      Drivers getting orphaned clusters in Atlas due to these tests.

      How does this affect the end user?

      Can incur high Atlas costs - for example the Node team hit the monthly limit.

      How likely is it that this problem or use case will occur?

      Regularly

      If the problem does occur, what are the consequences and how severe are they?

      Once driver team cluster limit is hit, no more clusters can be created and subsequent tests will always fail.

      Is this issue urgent?

      For Node, yes.

      Is this ticket required by a downstream team?

      No

      Is this ticket only for tests?

      Test improvements

      Acceptance Criteria

      • Reorganize the lambda scripts in drivers-evergreen-tools
      • Update the spec recommendations to use task groups
      • Show working prototype CI in 1 driver

            Assignee:
            durran.jordan@mongodb.com Durran Jordan
            Reporter:
            durran.jordan@mongodb.com Durran Jordan
            Andreas Braun Andreas Braun
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: