Implement platform-conditional Bazel Python toolchain registration to fix Windows builds

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 8.3.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • DevProd Correctness
    • Fully Compatible
    • Correctness 2026-01-12
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      ISSUE

        Bazel unconditionally downloads and registers Python toolchains for all 7 platforms (Linux x86{_}64, ARM64, PPC64LE, s390x, macOS x8664, ARM64, Windows x8664) regardless of the host platform. This causes:

       

        1. Windows Build Failures (CRITICAL):

           - Linux Python toolchains contain symlinks in terminfo database

           - Windows NTFS cannot extract these symlinks

           - Bazel analysis phase fails with "ERROR: circular symlinks detected"

           - Affects 3 Windows variants: windows-compile-required, windows-2022-compile-required, windows-2022

       

        2. Bandwidth and Storage Waste (HIGH):

           - All platforms download ~729MB of toolchains

           - Each platform only needs ~90-120MB (their specific toolchain)

           - 88-93% waste per build

           - ~22GB unnecessary downloads per full CI run

           - Affects 32 Linux/macOS variants

       

        ROOT CAUSE

        In MODULE.bazel (lines 226-234), all 7 Python toolchains are registered unconditionally:

       

        register{_}toolchains(

            "@py_linux_arm64//:python_toolchain",

            "@py_linux_ppc64le//:python_toolchain",

            "@py_linux_s390x//:python_toolchain",

            "@py_linux_x86_64//:python_toolchain",

            "@py_macos_arm64//:python_toolchain",

            "@py_macos_x86_64//:python_toolchain",

            "@py_windows_x86_64//:python_toolchain",

        )

       

        Bazel's module extension system downloads and evaluates all registered toolchains during the analysis phase, even if they will never be used on the current platform.

       

        IMPACT

        - 3 Windows variants: Build failures (cannot proceed)

        - 32 Linux/macOS variants: 80-93% bandwidth/storage waste

        - Total affected: 35 build variants across all platforms

       

        SOLUTION

        Implement platform-conditional toolchain registration in bazel/toolchains/python/python{_}toolchain.bzl:

       

        1. Detect host platform in module extension context (os.name, os.arch)

        2. Create stub repositories for non-matching platforms

           - Stub repos satisfy MODULE.bazel's register{_}toolchains() requirements

           - Use impossible constraint (@platforms//:incompatible) so never selected

           - No actual downloads occur for stub repos

        3. Only create full toolchain repository for matching host platform

       

        This ensures each platform only downloads the single toolchain it needs.

            Assignee:
            Nicholas Jefferies
            Reporter:
            Nicholas Jefferies
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: