Download only the multiversion binaries a task needs

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Fixed
    • Priority: Minor - P4
    • 9.1.0-rc0, 9.1.0-rc1023
    • Affects Version/s: None
    • Component/s: None
    • None
    • DevProd Test Infrastructure
    • Fully Compatible
    • DevProd Test Infra 2026-09-22
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Problem

      select_multiversion_binaries runs once per build variant and resolves every version listed in
      evergreen/multiversion_selection.sh – currently last-lts, last-continuous, 7.0, 8.0 and
      8.0.16 – into multiversion-downloads.json. Every multiversion task on that variant then hands
      that file to db-contrib-tool setup-repro-env, which downloads all of them. Passing the tool a
      .json path makes it a single VERSIONS_FILE request that it expands wholesale; there is no way
      to ask for a subset.

      So a jstestfuzz task testing only against last-LTS also downloads 7.0, 8.0 and 8.0.16. Only the
      multiversion and multiversion_auth suites need those, via major_version_upgrade.js and
      skip_level_upgrade.js, which walk every supported release. Nothing is cached between tasks
      either, since multiversion_setup.sh removes the install and link dirs first.

      Change

      Trim multiversion-downloads.json to what the task needs before db-contrib-tool sees it, in
      buildscripts/multiversion_downloads_filter.py. Two cases, no inference:

      • the task generator passed the versions this task needs -> keep those entries, drop the rest
      • it passed nothing -> leave the file alone and download everything, as today

      The versions come from multiversion_setup_versions, added to the do multiversion setup
      function call by mongo-task-generator (see linked ticket). Until a generator release that sets it
      is pinned in evergreen/prelude_mongo_task_generator.sh, every task takes the second branch and
      behaviour is unchanged.

      Result

      Measured on replica_sets_jscore_passthrough_last_lts_new_new_old: one binary instead of four,
      and the multiversion setup step drops from ~38s to ~17s. Suites that test against several
      versions (no_multiversion_generate_tasks) are unaffected and still download everything.

      Testing

      Evergreen patches covering both paths on enterprise-rhel-8-64-bit-multiversion and
      enterprise-rhel-8-64-bit-large-txns-format: fan-out tasks trim to one binary, opt-out tasks
      download all four, tests pass in both. 20 unit tests, run by buildscripts_test.

            Assignee:
            Nicholas Jefferies
            Reporter:
            Nicholas Jefferies
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: