Uploaded image for project: 'Drivers'
  1. Drivers
  2. DRIVERS-2497

Remove use of activate_venv.sh and utils.sh

XMLWordPrintableJSON

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Unknown Unknown
    • None
    • Component/s: None
    • None
    • Needed
    • Hide

      Note: the new venv-utils.sh, activate-kmstlsvenv.sh, activate-authawsvenv.sh and activate-ocspvenv.sh scripts MUST be sourced in a Bash shell. See DRIVERS-2497 description and comments for details.

      Show
      Sync Drivers Evergreen Tools repo with a3a17f11 (formerly e276a93e and f5d1c049 ). Drivers MUST replace use of utils.sh with venv-utils.sh . Drivers MUST replace use of csfle/activate_venv.sh with csfle/activate-kmstlsvenv.sh . Drivers MUST replace use of auth_aws/activate_venv.sh with auth_aws/activate-authawsvenv.sh . Drivers SHOULD use the new ocsp/activate-ocspvenv.sh . Drivers SHOULD replace ad hoc venv-capable Python binary search routines with find-python3.sh . Note: the new venv-utils.sh , activate-kmstlsvenv.sh , activate-authawsvenv.sh and activate-ocspvenv.sh scripts MUST be sourced in a Bash shell. See DRIVERS-2497 description and comments for details.
    • $i18n.getText("admin.common.words.hide")
      Key Status/Resolution FixVersion
      CDRIVER-4519 Fixed 1.24.0
      CXX-2612 Fixed 3.8.0
      CSHARP-4399 Fixed 2.19.0
      GODRIVER-2643 Done
      JAVA-4806 Fixed 4.9.0
      NODE-4800 Fixed 4.13.0
      MOTOR-1064 Gone away
      PYTHON-3507 Gone away
      PHPLIB-1038 Fixed 1.16.0
      RUBY-3175 Fixed 2.19.0
      RUST-1537 Fixed 2.4.0
      SWIFT-1678 Won't Do
      PHPC-2164 Fixed 1.16.0
      $i18n.getText("admin.common.words.show")
      #scriptField, #scriptField *{ border: 1px solid black; } #scriptField{ border-collapse: collapse; } #scriptField td { text-align: center; /* Center-align text in table cells */ } #scriptField td.key { text-align: left; /* Left-align text in the Key column */ } #scriptField a { text-decoration: none; /* Remove underlines from links */ border: none; /* Remove border from links */ } /* Add green background color to cells with FixVersion */ #scriptField td.hasFixVersion { background-color: #00FF00; /* Green color code */ } /* Center-align the first row headers */ #scriptField th { text-align: center; } Key Status/Resolution FixVersion CDRIVER-4519 Fixed 1.24.0 CXX-2612 Fixed 3.8.0 CSHARP-4399 Fixed 2.19.0 GODRIVER-2643 Done JAVA-4806 Fixed 4.9.0 NODE-4800 Fixed 4.13.0 MOTOR-1064 Gone away PYTHON-3507 Gone away PHPLIB-1038 Fixed 1.16.0 RUBY-3175 Fixed 2.19.0 RUST-1537 Fixed 2.4.0 SWIFT-1678 Won't Do PHPC-2164 Fixed 1.16.0

      Summary

      The utils.sh and activate_venv.sh scripts provided by Drivers Evergreen Tools are deprecated and will eventually be removed. Drivers must replace their use with the new venv-utils.sh, activate-kmstlsvenv.sh, and find-python3.sh scripts instead.

      The new venv-utils.sh and activate-kmstlsvenv.sh are hygienic alternatives to utils.sh and activate_venv.sh that do not leak variables into the environment and do not depend on environment variables for their behavior. Both of these old script behaviors has been observed to cause unexpected and often difficult-to-diagnose distro-specific task failures.

      Furthermore, the existing built-in logic to detect a viable python binary capable of creating a virtual environment has been observed to fail on several distros both old and new. To facilitate a more consistent and comprehensive method for finding a viable python binary, a new find-python3.sh script has been provided to replace the current ad hoc implementations both in Drivers and in Drivers Evergreen Tools.

      Before:

      cd drivers-evergreen-tools/.evergreen/csfle
. ./activate_venv.sh # Leaks PYTHON set prior to venv.
. ./set-temp-creds.sh # Uses leaked non-venv PYTHON.

      After:

      cd drivers-evergreen-tools/.evergreen/csfle
. ./activate-kmstlsvenv.sh # No leaks.
. ./set-temp-creds.sh # Uses venv python.

      Before:

      . drivers-evergreen-tools/.evergreen/utils.sh
PYTHON="path/to/python" # Ad hoc, complicated, and inconsistent.
venvcreate "$PYTHON" venv # Leaks PYTHON, depends on undocumented OS env var.

      After:

      . drivers-evergreen-tools/.evergreen/venv-utils.sh
. drivers-evergreen-tools/.evergreen/find-python3.sh
PYTHON="$(find_python3)" # Uniform python lookup utility.
venvcreate "$PYTHON" venv # No leaks or undocumented dependencies.

      Motivation

      Who is the affected end user?

      Drivers that are using the utils.sh or activate_venv.sh scripts provided by Drivers Evergreen Tools.

      How does this affect the end user?

      Drivers must update their scripts to replace usage of utils.sh or activate_venv.sh as described above. This should only affect Evergreen configurations and scripts; Drivers implementations should not require any modifications.

      How likely is it that this problem or use case will occur?

      Depends on the Driver's use of the scripts and related environment variables such as PYTHON, PYTHON_BINARY, and OS.

      If the problem does occur, what are the consequences and how severe are they?

      Time lost to diagnosing unexpected and problematic Python virtual environment creation or activation behavior.

      Is this issue urgent?

      No.

      Is this ticket required by a downstream team?

      No.

      Is this ticket only for tests?

      Yes. Evergreen configuration and scripts only.

            Assignee:
            ezra.chung@mongodb.com Ezra Chung
            Reporter:
            ezra.chung@mongodb.com Ezra Chung
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: