ExportXMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Build
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Builds custom builds (Splunk on 9.0/8.0/7.0, MCAS on 6.0/5.0/4.4) at each GA release tag, then publishes them to the public mdb-build-public bucket once https://downloads.mongodb.org/full.json lists the release. The official release pipeline is unchanged.

      Why

      • GA tags sit on vX.Y.Z-hotfix branches that Evergreen never builds. Only 2 of 20 GA releases since May got a git-tag version, and those are created inactive (waterfall_disabled). The git-tag release channel therefore almost never fires.
      • full.json is updated days to weeks after the tag. A ~2h build started at tag time is done long before the release.

      How

      A new hourly task, custom_build_release_poll, runs on custom-build-release-poller in mongodb-mongo-master-nightly (cron: "0 * * * *", patchable: false). It runs buildscripts/private/custom_builds/release_poller.py. For each GA tag that is the newest release of its series, or newer:

      1. Tag → build. If the tag's tree defines custom build variants, the poller creates one patch on mongodb-mongo-vX.Y with patch-file --base <tag commit> --allow-empty. The patch runs their custom_build_deliver with custom_build_base_version=X.Y.Z and is_release=true, and delivers to the existing patch channel with its manifest. No Splunk branch changes are needed: the current hotfix branches (v9.0.5, v8.0.35, v7.0.46) already have everything this uses.
      2. full.json → publish. Once full.json lists X.Y.Z at the tag commit and every deliver task succeeded, the poller:
        • checks each manifest's version and revision and each tarball's sha256, verifying every variant before uploading any;
        • copies the tarball and a public manifest with public-read to mdb-build-public/custom-builds/<custom_build_upload_path_prefix>/<version>/.

      Per-tag state (build patch ID, published) is s3://mdb-build-private/custom-build-releases/<tag>.json. Before creating any patch, the poller confirms the state can be read back, so a permissions problem can't turn into a new patch every hour. A failed deliver task fails the poll task until someone reruns it in Spruce.

      Credentials are existing project variables only:

      • evergreen_api_user/evergreen_api_key, through the existing configure evergreen api credentials function;
      • a GitHub token (contents:read) from the existing generate github token function, for tags and tag-tree files;
      • aws_key_build/aws_secret_build.

      Testing

      • Unit tests (bazel test //buildscripts/private/custom_builds:all) cover tag selection, variant discovery on both config layouts, the state transitions, and refusing mismatched or tampered builds. They also check that a release is published once and a patch is created once.
      • Live read-only check against today's tags, full.json and tag trees: it selects r9.0.4, r8.0.34 and r7.0.45 (Splunk) and skips the pre-framework EOL tags.
      • The tag-time build mechanics are being verified now. Patch 6ad7d884667c3a00074db2df on mongodb-mongo-v8.0 was created exactly as the poller creates it, with base = the off-branch r8.0.34 tag commit.
      • evergreen validate on etc/evergreen_nightly.yml and the master YAML lint pass.

            Assignee:
            Zack Winter
            Reporter:
            Zack Winter
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: