Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-4069

PyMongo Evergreen and Script Running Refactor

XMLWordPrintableJSON

    • Type: Icon: Epic Epic
    • Resolution: Done
    • Priority: Icon: Unknown Unknown
    • None
    • Affects Version/s: None
    • Component/s: None
    • Evergreen scripts hygiene
    • Python Drivers
    • Not Needed
    • Done
    • 6
    • 3
    • 6
    • 100
    • None
    • Hide

      Engineer(s): Steve, Jib, Noah
      2025-04-10

      • Completed over last 2 weeks
        • Completed conversion of all tasks and variants
        • Began conversion of functions, blocked on a PR to shrub.py, which will close out the last ticket in the Epic.
      • Focus over next two weeks
        • Finish the last ticket in the Epic.

      Engineer(s): Steve, Jib, Noah
      2025-03-14

      • Completed over last 2 weeks
        • Added "Current" version of Python to the toolchain and used it drivers-tools and pymongo
        • Converted several test suites to be runnable locally using the new test scripts. All of our complicated tests are converted.
      • Focus over next two weeks
        • Finalize the test suite migration

      Engineer(s): Steve, Jib, Noah
      2025-03-27

      • Completed over last 2 weeks
        • Completed conversion of all test suites to the simplified test runner
      • Focus over next two weeks
        • Convert remaining evergreen config to be generated
        • Reduce the number of variants by moving python version and mongodb version to the task level where appropriate.

      Engineer(s): Steve, Jib, Noah
      2025-03-3

      • Completed over last 2 weeks
        • Added just and uv to python toolchain
        • Created python clis for test setup and run
      • Focus over next two weeks
        • Continue to refine the test runner
        • Continue to convert test scenarios to use the test runner

      Engineer(s): Steve, Jib, Noah
      2025-01-31

      • Completed over last 2 weeks
        • Updated to use just and uv
        • Converted bash scripts to Python for easier maintenance, improvements, and debuggability
      • Focus over next two weeks
        • Merge PR for test runner improvements

      Engineer(s): Steve, Jib, Noah
      2025-01-03

      • Completed over last 2 weeks
        • On hold til Jan
      • Focus over next two weeks
        •  

      2024-12-10

      • Completed over last 2 weeks
        • Completed migration to subprocess over shell in evergreen
        • Migrated to ensuring we always use Rust and Hatch on Evergreen builds
      • Focus over next two weeks
        • On hold til Jan, will define scope more clearly
        • Currently not an active priority, is QOL picked up in Greenerbuild
      Show
      Engineer(s): Steve, Jib, Noah 2025-04-10 Completed over last 2 weeks Completed conversion of all tasks and variants Began conversion of functions, blocked on a PR to shrub.py, which will close out the last ticket in the Epic. Focus over next two weeks Finish the last ticket in the Epic. Engineer(s): Steve, Jib, Noah 2025-03-14 Completed over last 2 weeks Added "Current" version of Python to the toolchain and used it drivers-tools and pymongo Converted several test suites to be runnable locally using the new test scripts. All of our complicated tests are converted. Focus over next two weeks Finalize the test suite migration Engineer(s): Steve, Jib, Noah 2025-03-27 Completed over last 2 weeks Completed conversion of all test suites to the simplified test runner Focus over next two weeks Convert remaining evergreen config to be generated Reduce the number of variants by moving python version and mongodb version to the task level where appropriate. Engineer(s): Steve, Jib, Noah 2025-03-3 Completed over last 2 weeks Added just and uv to python toolchain Created python clis for test setup and run Focus over next two weeks Continue to refine the test runner Continue to convert test scenarios to use the test runner Engineer(s): Steve, Jib, Noah 2025-01-31 Completed over last 2 weeks Updated to use just and uv Converted bash scripts to Python for easier maintenance, improvements, and debuggability Focus over next two weeks Merge PR for test runner improvements Engineer(s): Steve, Jib, Noah 2025-01-03 Completed over last 2 weeks On hold til Jan Focus over next two weeks   2024-12-10 Completed over last 2 weeks Completed migration to subprocess over shell in evergreen Migrated to ensuring we always use Rust and Hatch on Evergreen builds Focus over next two weeks On hold til Jan, will define scope more clearly Currently not an active priority, is QOL picked up in Greenerbuild

      Context

      The EVG docs https://docs.devprod.prod.corp.mongodb.com/evergreen/Project-Configuration/Best-Practices#subprocessexec give several reasons why it should be preferred to call out to scripts instead of putting the steps in the config file.

      We should additionally structure the scripts so they can be called locally and on spawn hosts. See https://docs.google.com/document/d/1d20ZRYsDn6-8IvPUMXcDLbp1fXk_ktW11fpxzN4SIvA/edit?tab=t.0 for discussion.

      Definition of done

      Scripts are restructured and can be run locally and on a spawn host.

      Pitfalls

      The EVG directory could get a bit cluttered, we might want to introduce sub folders.

        1.
        		 Remove ${PREPARE_SHELL} Calls PYTHON-4631 Sub-task Closed Jib Adegunloye  
        2.
        		 Create individualized scripts for all shell.exec commands PYTHON-4721 Sub-task Closed Noah Stapp  
        3.
        		 Consolidate startup and teardown tasks PYTHON-4965 Sub-task Closed Noah Stapp  

            Assignee:
            noah.stapp@mongodb.com Noah Stapp
            Reporter:
            steve.silvester@mongodb.com Steve Silvester
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved:
              None
              None