Overview

Deliver the macOS portion of Milestone 2 of the hermetic RBE container project: enable macOS builds to use pinned Linux containers for cross-compilation and remote execution, while running native macOS tests and packaging steps on the host.

Parent Scope

• Epic: DEVPROD-33617
• Milestone: Windows/macOS containerized cross-compilation and RBE
• Windows counterpart: SERVER-133629

Motivation

macOS builds depend on platform-specific build hosts and host-installed toolchains. Moving compilation and eligible build actions into pinned Linux hermetic containers reduces differences between local, Evergreen, and RBE execution and expands the build graph eligible for remote execution.

Scope of Work

• Enable Linux-container cross-compilation for arm64 and x86_64 macOS targets using -config=macos-cross-arm64 and -config=macos-cross-x86_64.
• Route compile, IDL, and platform-independent generation actions to Linux ARM64 RBE; route local C++ link, archive, and strip actions through a persistent Linux container.
• Keep native macOS helpers, dsymutil, installation/packaging, and test execution on the macOS host, including native tests within the same Bazel invocation and host bazel run support.
• Pin the container image, macOS SDK, and cross toolchain. Align the Linux ARM64 cross toolchain with the native macOS LLVM version: LLVM 19.1.7 from the official GitHub release, with the pinned MacOSX15.2.sdk.
• Ensure declared action inputs, debug-info dependencies, output downloads, runfiles, and shared filesystem mounts work across RBE, the Linux container, and the native host.
• Provision and validate a Docker-compatible runtime on Evergreen macOS hosts, including daemon startup, build-user access, and shared mount visibility.
• Add Evergreen coverage for supported macOS variants and document setup, default opt-in/overrides, execution strategies, and container/toolchain/SDK maintenance.

Acceptance Criteria

• arm64 and x86_64 macOS cross-compilation completes from supported macOS hosts using the pinned hermetic Linux image.
• Compile, IDL, link, archive, strip, debug-info, installation, and packaging actions use the intended local-container, RBE, or native-host strategy.
• Produced Mach-O binaries execute natively on macOS, and test results are reported correctly in Evergreen.
• Distribution and debug archives, signing, and artifact publication complete successfully.
• Local-container and remote execution use consistent pinned toolchain and SDK inputs.
• Evergreen master/nightly coverage is enabled for the supported macOS variants.
• Runtime setup and image/toolchain/SDK update, signing, and rollback procedures are documented.
• Failures clearly identify missing or unsupported runtimes, toolchains, SDKs, and shared mounts.

Current Validation

Evergreen patch 7649 validates the enterprise-macos-arm64 path:

• Nine tasks passed, including archive_dist_test, archive_dist_test_debug, unit_tests (468/468 test entries), bazel_run_lint, and run_bazel_program.
• Distribution compilation, strip/install, debug archives, signing, and packaging passed.
• crypt_create_lib compiled and archived successfully, then failed during a macnotary signing upload URL request with HTTP 500; this is the known infrastructure issue BF-46601.
• Full x86_64 Evergreen validation and master/nightly rollout remain to be completed; the passing ARM64 patch does not establish those criteria.

Risks and Open Questions

• Complete Docker-compatible runtime provisioning and startup reliability on supported macOS distro images; the current CI path has been tested with Colima.
• Confirm shared mount and downloaded-input visibility across the native host, Linux container, and RBE.
• Validate x86_64 target coverage and identify any remaining native-host-only actions.
• Confirm RBE capacity/cost and supported opt-out behavior when a local container runtime is unavailable.

Related Work

• DEVPROD-41587 (formerly SERVER-133625) tracks installing a Docker-compatible runtime on Evergreen macOS distro images.
• SERVER-133629 tracks the Windows counterpart.
• Setup and execution strategy documentation: bazel/docs/macos_hermetic_container_cross.md.

Non-Goal

This milestone does not attempt to make builds fully reproducible.