Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-55129

virtualenv setup in evergreen takes too long

    • Fully Compatible
    • v5.0
    • STM 2021-08-23, STM 2021-09-06
    • 2

      This compile task took 3:22, but of that time, the actual compile work took only 2:18: https://evergreen.mongodb.com/task_log_raw/mongodb_mongo_master_enterprise_rhel_80_64_bit_dynamic_required_compile_dist_test_6ddf0cc7c320987e801e24bf171147d1fd7a0a5b_21_03_07_11_05_12/0?type=T#L11831

      We spent nearly 30 seconds preparing the virtualenv: https://evergreen.mongodb.com/task_log_raw/mongodb_mongo_master_enterprise_rhel_80_64_bit_dynamic_required_compile_dist_test_6ddf0cc7c320987e801e24bf171147d1fd7a0a5b_21_03_07_11_05_12/0?type=T#L56

      That's roughly 15% of the task time(we spent about the same 15% cloning the repo).

      Overall, we run about 600 tasks for each build, so that's 300 minutes or 5 hours of compute time spent creating the same virtualenv. For the build above, that represents about 4% of our total compute time.

      We should find a way to construct these virtualenvs faster.

            Assignee:
            mikhail.shchatko@mongodb.com Mikhail Shchatko
            Reporter:
            andrew.morrow@mongodb.com Andrew Morrow (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: