-
Type:
Bug
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
-
DevProd Test Infrastructure
-
Fully Compatible
-
ALL
-
200
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Overview
The enterprise-rhel-8-64-bit-future-git-tag-multiversion variant has been failing at 100% since 2026-05-16 (BF-43386) with task timeouts in replica_sets_multiversion.
Root Cause
The replica_sets_multiversion_gen task generates replica_sets_last_continuous and replica_sets_last_lts suites, both of which use useRandomBinVersionsWithinReplicaSet. In this variant the "latest" binary is compiled at FCV 100.0 (via bv_future_git_tag: r100.0.0-9999), while "last-continuous" is 9.0 and "last-lts" is 8.0.
When a 9.0 or 8.0 secondary needs to initial sync from a 100.0 primary, it reads FCV 100.0 from admin.system.version and fatally asserts because 100.0 is not in its valid FCV set:
BadValue: Invalid feature compatibility version value '100.0'. Expected one of the following versions: '9.0', '8.3', '8.0', ... Fatal assertion 40088 at src/mongo/db/repl/replication_coordinator_impl.cpp:937
This is a fundamental incompatibility: older binaries can never initial sync from a cluster running at a future FCV. The analogous multiversion_gen task (same file) was already tagged future_git_tag_incompatible for the same reason; replica_sets_multiversion_gen was missed.
The failures were masked before SERVER-120949 (2026-04-28) because the compile was broken and the "future" primary was not actually compiled at 100.0. Once SERVER-120949 was fully fixed (2026-05-07), the primary correctly compiles at 100.0, exposing this gap.
Scope of Work
- etc/evergreen_yml_components/tasks/resmoke/server_divisions/durable_transactions_and_availability/tasks.yml – add "future_git_tag_incompatible" to the replica_sets_multiversion_gen tags list, mirroring multiversion_gen
Acceptance Criteria
- enterprise-rhel-8-64-bit-future-git-tag-multiversion variant no longer runs replica_sets_multiversion task
- BF-43386 stops recurring
Technical Notes
- The variant task selector - name: .multiversion !.future_git_tag_incompatible automatically excludes any task tagged future_git_tag_incompatible
- No server code changes needed – CI configuration only
- Related: SERVER-120949 (the compile-side fix that exposed this test-side gap)
- is related to
-
SERVER-120949 Remove Tag Based FCV Generation
-
- Closed
-