-
Type:
Task
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: Bazel
-
None
-
Fully Compatible
-
Build OnDeck
-
200
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Since SERVER-135110, the s390x release variants (`enterprise-rhel-
s390x`) run `archive_dist_test` on ARM64 hosts. On the waterfall (non-patch, `release_rbe: false`), `bazel_compile.sh` enables release-local mode (`config=public-release-local`). `_linux_cross_rbe_local_host_args` then switched the build to the native `//bazel/platforms:<distro>_s390x` platform and cleared `-extra_toolchains`, which assumes a native zSeries host. On an ARM64 host there is no s390x-targeting toolchain, so analysis failed:
```
No matching toolchains found for types: @@bazel_tools//tools/cpp:toolchain_type
```
Patch builds never enter release-local mode, so they passed.
*Changes*
- `bazel/wrapper_hook/hermetic_container/cross/linux.py`: adds `_linux_cross_release_local_on_exec_host`, which checks whether the host architecture matches the config's `exec_arch`, and `_linux_cross_release_local_exec_host_args`. The latter keeps the cross-RBE platforms, toolchain selectors and protobuf/gRPC module overlays, then forces a fully local build: `-
remote_executor=`, `spawn_strategy=local`, `remote_download_outputs=all`, `noremote_accept_cached`, `remote_upload_local_results=false`, `-modify_execution_info=.*=+no-cache`. Evergreen's release-local safety flags stay at the end of the command line, as the provenance check requires. - `bazel/wrapper_hook/hermetic_container/runner.py`: release-local on the execution-architecture host uses the new args with `force_local=True`, so every action runs in the host-native container. Release-local on native IBM hosts is unchanged.
- `tools/bazel`: `linux_cross_rbe_exec_arch_from_args` keeps the cross selectors for release-local when `uname -m` matches the config's execution architecture. Before, it always injected cleared `MONGO_LINUX_CROSS_TOOLCHAIN`/`MONGO_WASI_SDK_EXEC_ARCH` repo_env values before the release suffix. Those overrode the wrapper's selectors, so the cross toolchain repository became a no-op. The wrapper hook also repeats the selectors in its final option block so they always win.
- `bazel/docs/linux_hermetic_container_cross_rbe.md`: documents both release-local modes.
*Safety*
- Release execution logs still contain no remote spawns or cache hits.
- Behavior on native zSeries/ppc64le hosts and all non-release (cross-RBE) builds is unchanged.
*Testing*
- New `test_run_hermetic_container_release_cross_build_on_exec_host_stays_cross` in `cross/windows_test.py` uses the failing task's exact config. It fails without the fix, reproducing the waterfall arguments, and passes with it. The existing native release-local test now pins the host arch to `s390x`.
- The `windows`, `macos`, `env`, `config` and `distro` test suites pass. The 6 failures in `linux_test`/`runner_test` are identical on the base commit (macOS-local environment).
- `tools/bazel_wrapper_test.sh`: pins the host arch with a fake `uname` for the existing native release-local case, and adds an ARM64 exec-host release-local case.
- Evergreen patch forcing release-local on `enterprise-rhel-10-s390x` `archive_dist_test`, first commit only. The new path ran but was defeated by `tools/bazel`: https://spruce.corp.mongodb.com/version/6abac12cf5e7610007681ee5
- Re-run with both commits: https://spruce.corp.mongodb.com/version/6abac93d865345000756a3e7
- is related to
-
SERVER-135110 Move 390x CI builds off under-provisioned IBM machines via cross-compilation
-
- Closed
-