[MONGOCRYPT-437] Replace hard-coded values from remote_file Created: 02/Jun/22  Updated: 28/Oct/23  Resolved: 22/Jul/22

Status: Closed
Project: Libmongocrypt
Component/s: None
Affects Version/s: None
Fix Version/s: 1.6.0, 1.5.2, 1.6.0-alpha0

Type: Task Priority: Major - P3
Reporter: Roberto Sanchez Assignee: Roberto Sanchez
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to MONGOCRYPT-315 Upload tagged releases to a URL conta... Closed
Binding Changes: Not Needed

 Description   

The Evergreen project configuration contains several instances of hard-coded values in the remote_file parameter of s3.put commands. This was okay when there was a single Evergreen project associated with the repository and when releases were made directly from the master branch. However, we now have three Evergreen projects associated with the repository, releases made from release-specific branches, and a project specifically for tracking work done on feature branches.

The specific instances that need to be addressed are occurrences of libmongocrypt that should be replaced with ${project} and occurrences of master that should be replaced with ${branch_name}. These are located in the s3.put commands of the upload-java and upload-all tasks. Similar values in the windows-upload and debian-package-build tasks may not necessarily need to be changed, but rationale for hard-coded values should be included in a comment.

Changes to these values should be made in such a way as to ensure that workflows that depend on the affected artifacts (e.g., downstream driver projects) are not broken.



 Comments   
Comment by Githook User [ 22/Jul/22 ]

Author:

{'name': 'Roberto C. Sánchez', 'email': 'roberto@connexer.com', 'username': 'rcsanchez97'}

Message: MONGOCRYPT-437 remove hard-coded branch names in Evergreen s3.put targets (#421)
Branch: r1.5
https://github.com/mongodb/libmongocrypt/commit/2392b258b20a183b4a95e9701b4086ee390bbf52

Comment by Githook User [ 22/Jul/22 ]

Author:

{'name': 'Roberto C. Sánchez', 'email': 'roberto@connexer.com', 'username': 'rcsanchez97'}

Message: MONGOCRYPT-437 remove hard-coded branch names in Evergreen s3.put targets (#420)
Branch: master
https://github.com/mongodb/libmongocrypt/commit/8cbd0692409cb665a83072a55212528c19a3073f

Comment by Roberto Sanchez [ 09/Jul/22 ]

kevin.albertson@mongodb.com, at first glance this issue seemed simple and straightforward, and to a certain extent it is possible to simply make the changes I described in the initial summary and call it done. However, as I began to think about some of the implications associated with the assumptions and the proposed changes it seemed like a more thorough reevaluation would be useful. I apologize up front if this comment ends up lengthy, but I want to make sure to cover all the bases.

Given the way that libmongocrypt is handled in Evergreen, I question whether our artifact locations need the branch as part of the URL at all. The projects libmongocrypt (tracking the master branch) and libmongocrypt-release (tracking the latest release branch, currently r1.5) give downstream projects the ability to choose whether their own projects will depend upon artifacts which represent that absolute latest development work, or more stable released (or planned for release) changes, respectively. The changes implemented as part of MONGOCRYPT-315 make the latter category even more useful, as downstream projects can target artifacts based on a tag rather than only a commit SHA.

The libmongocrypt Evergreen project tracks only a single branch, master, and that seems unlikely to change. Since it is the project that represents the latest active development work, including master in the URL seems redundant. The libmongocrypt-release project will change the branch that it tracks as new minor releases are made and their corresponding branches are created. Including the branch name in the artifacts for the libmongocrypt-release project does help a person identify the release series of a particular artifact based only on the URL, but it also means that any project that wants to use "the latest artifact representing a release version or something that very likely will become a release version" must also be aware of new minor releases and change the artifact locations in their own projects whenever a new release branch is started.

It appears to me that we can create a simpler and more useful experience for downstream projects by dropping the branch name from the URLs and using the ${project} Evergreen expansion.

That will result in artifacts locations of the form:

${project}/all/latest/libmongocrypt-all.tar.gz
${project}/all/${commit}/libmongocrypt-all.tar.gz
${project}/all/${tag}/libmongocrypt-all.tar.gz
${project}/${build_variant}/latest/libmongocrypt.tar.gz
${project}/${build_variant}/${commit}/libmongocrypt.tar.gz
${project}/${build_variant}/${tag}/libmongocrypt.tar.gz
${project}/${build_variant}/latest/libmongocrypt-distro-packages.tar.gz
${project}/${build_variant}/${commit}/libmongocrypt-distro-packages.tar.gz
${project}/${build_variant}/${tag}/libmongocrypt-distro-packages.tar.gz

The value of ${project} would vary between libmongocrypt and libmongocrypt-release. This would give downstream projects easy access to an artifact with either all binary builds or only those pertaining to a particular build variant, whether based on a commit SHA, tag name, or latest, either for latest development or latest stable release. Particularly, this would ensure that the notion of "latest stable release" does not become stale because the branch tracked by the libmongocrypt-release project changed and a downstream project fails to update the URLs being used to retrieve artifacts. For any project that really wants "the actual stable release such and such", they can use tag-based URLs now and it would be incumbent upon the project in question to update URLs to point to a newer release according to their own development and configuration management policies.

Clearly, if we expand the scope of this ticket in the way I am suggestion, we will need to make sure that downstream users are aware of the change before it takes place. Otherwise, they will be stuck with stale artifacts. One other thing to note: given that 1.5.0 was just released and the r1.5 branch just created, I think it makes sense to leave the artifact locations as they are for that branch. The changes I am proposing would be only for master now, and then for r1.6 and later release branches.

Is this something that seems worthwhile? If so, what is the best way to ensure that everyone who might be affected is warned appropriately?

Generated at Thu Feb 08 09:08:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.