Tag replica_sets_multiversion_gen as future_git_tag_incompatible to fix enterprise-rhel-8-64-bit-future-git-tag-multiversion failures

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.0.0-rc1
    • Affects Version/s: None
    • Component/s: None
    • None
    • DevProd Test Infrastructure
    • Fully Compatible
    • ALL
    • 200
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Overview

      The enterprise-rhel-8-64-bit-future-git-tag-multiversion variant has been failing at 100% since 2026-05-16 (BF-43386) with task timeouts in replica_sets_multiversion.

      Root Cause

      The replica_sets_multiversion_gen task generates replica_sets_last_continuous and replica_sets_last_lts suites, both of which use useRandomBinVersionsWithinReplicaSet. In this variant the "latest" binary is compiled at FCV 100.0 (via bv_future_git_tag: r100.0.0-9999), while "last-continuous" is 9.0 and "last-lts" is 8.0.

      When a 9.0 or 8.0 secondary needs to initial sync from a 100.0 primary, it reads FCV 100.0 from admin.system.version and fatally asserts because 100.0 is not in its valid FCV set:

      BadValue: Invalid feature compatibility version value '100.0'.
      Expected one of the following versions: '9.0', '8.3', '8.0', ...
      Fatal assertion 40088 at src/mongo/db/repl/replication_coordinator_impl.cpp:937
      

      This is a fundamental incompatibility: older binaries can never initial sync from a cluster running at a future FCV. The analogous multiversion_gen task (same file) was already tagged future_git_tag_incompatible for the same reason; replica_sets_multiversion_gen was missed.

      The failures were masked before SERVER-120949 (2026-04-28) because the compile was broken and the "future" primary was not actually compiled at 100.0. Once SERVER-120949 was fully fixed (2026-05-07), the primary correctly compiles at 100.0, exposing this gap.

      Scope of Work

      • etc/evergreen_yml_components/tasks/resmoke/server_divisions/durable_transactions_and_availability/tasks.yml – add "future_git_tag_incompatible" to the replica_sets_multiversion_gen tags list, mirroring multiversion_gen

      Acceptance Criteria

      • enterprise-rhel-8-64-bit-future-git-tag-multiversion variant no longer runs replica_sets_multiversion task
      • BF-43386 stops recurring

      Technical Notes

      • The variant task selector - name: .multiversion !.future_git_tag_incompatible automatically excludes any task tagged future_git_tag_incompatible
      • No server code changes needed – CI configuration only
      • Related: SERVER-120949 (the compile-side fix that exposed this test-side gap)

            Assignee:
            Steve McClure
            Reporter:
            Steve McClure
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: