Align Windows e2e testing with the platform support policy

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • None
    • Developer Tools

      Follow-up to COMPASS-9087, which added a time-to-first-query e2e smoke against the packaged app on the minimum supported version of each platform (Ubuntu 20.04, RHEL 8.0, macOS 13). Windows was left out.

      Investigation showed the original framing ("add e2e test on minimum Windows platform") is a no-op as written, but that our Windows coverage is not actually aligned with the support policy.

      Verified state of every Windows image in Evergreen

      Image Build OS
      windows-vsCurrent 17763 Server 2019
      windows64-vs2017 14393 Server 2016
      windows64-vs2019 14393 Server 2016
      windows-2022 20348 Server 2022
      windows-2022-dc 20348 Server 2022
      windows-2022-latest 20348 Server 2022
      windows2022-serverteam-1.0 20348 Server 2022
      windows64-vsMulti 20348 Server 2022

      Notes:

      • Evergreen offers only Windows Server images, so client versions are approximated by matching build numbers.
      • The vsXXXX in a distro name is the Visual Studio version, not the Windows version: windows64-vs2019 is Server 2016, an Server 2019 despite the name.
      • windows-vsCurrent (17763) is the only image matching our minimum supported Windows (10, build 17763), and test-packaged-app-windows already runs there. But vsCurrent  is a rolling image by design, so that match is incidental: if DevProd rebases it onto a newer Server, minimum-version coverage disappears silently.
      • No image reports 22000 (Win11 21H2) or 26100 (Win11 24H2 / Server 2025), so Windows 11 is currently untestable. 20348 sits between the two a proxy for either.

      Scope

      Item 1 — Windows 10 (minimum supported) e2e. Add a packaged-app e2e variant running the time-to-first-query smoke on a pinned Windows 10 (build 17763) image, following the pattern of the existing minimum-version variants from COMPASS-9087. Repoint or retire test-packaged-app-windows, which currently provides this coverage only by coincidence of vsCurrent's base image.

      Item 2 — Windows 11 e2e. Add the equivalent variant on a pinned Windows 11 (build 26100) image. This is new coverage; we have none today.

      Both items depend on DevProd providing the images — see below. Neither can be implemented against the images available now: the only candidate for item 1 is a rolling image, and there is no candidate at all for item 2.

      Dependencies

      Blocked on DevProd providing pinned images. Request covers a Windows 10 / Server 2019 (17763) image for item 1 and a Windows 11 / Server 2025 (26100) image for item 2. Also asks whether exact client versions could be virtualized on existing hosts, which would be preferable to approximating client Windows with Server for both items.

      Validation of the Server-as-substitute-for-client assumption. Does not block either item, but if Server images turn out to be a poor stand-in for client Windows, the approach above needs revisiting — as does COMPASS-9087's existing Linux and macOS coverage.

            Assignee:
            Ivan Medina
            Reporter:
            Ivan Medina
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: