[CDRIVER-2831] Migrate to dynamic VERSION_CURRENT Created: 21/Sep/18  Updated: 28/Oct/23  Resolved: 27/Sep/18

Status: Closed
Project: C Driver
Component/s: libbson, libmongoc
Affects Version/s: None
Fix Version/s: 1.14.0

Type: Improvement Priority: Major - P3
Reporter: Roberto Sanchez Assignee: Roberto Sanchez
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Epic Link: Automate C driver release

 Description   

The application version is determined at build time by looking in VERSION_CURRENT. That file is maintained manually. In an effort to move toward a more automated process, this file should be generated by a script (using the same approach described in CXX-584).

jesse, kevin.albertson, as part of this I would like to eliminate VERSION_RELEASED entirely. The only use of it I found was in assembling the URL for GitHub tarball download. Is there an alternative approach that would allow eliminating it?



 Comments   
Comment by Githook User [ 19/Feb/19 ]

Author:

{'name': 'Kevin Albertson', 'username': 'kevinAlbs', 'email': 'kevin.albertson@mongodb.com'}

Message: CDRIVER-2831 don't use --sort
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/f7353bc796d4bf98a43d1ea7d317b09173cc00a4

Comment by Githook User [ 28/Sep/18 ]

Author:

{'name': 'Roberto C. Sánchez', 'email': 'roberto@connexer.com', 'username': 'rcsanchez97'}

Message: CDRIVER-2831 migrate to dynamic VERSION_CURRENT, 2
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/55353fc040ef8202fcda0b56e2545536b640d340

Comment by Githook User [ 27/Sep/18 ]

Author:

{'name': 'Roberto C. Sánchez', 'email': 'roberto@connexer.com', 'username': 'rcsanchez97'}

Message: CDRIVER-2831 migrate to dynamic VERSION_CURRENT
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/f5b49120348218d803e8b9ad9c83f21127a4b344

Comment by A. Jesse Jiryu Davis [ 25/Sep/18 ]

We're following this spec:

https://github.com/mongodb/specifications/blob/0cf00e6ec2ebc1f8959d04c674623431b58198f1/source/mongodb-handshake/handshake.rst

For diagnosis, first try a printf in _mongoc_handshake_build_doc_with_application where doc->len > HANDSHAKE_MAX_SIZE, see what the handshake's JSON is so far before the function returns false.

In test_mongoc_handshake_data_append_success, if any of the strstr checks fail, printf before aborting.

The server does enforce a 512-byte limit if we don't, so we can't increase the limit client-side. Once diagnosed, I liked your proposed workaround of just setting the handshake metadata version number to something like "1.14.0-pre", without the extensive git info in the string. If that's hard for some reason, I propose a workaround with a private API used for testing that forces the version number to 1.2.3. 

Comment by Roberto Sanchez [ 25/Sep/18 ]

jesse, kevin.albertson, so I have run into a problem that is keeping me from being able to get this fully nailed down. Specifically I encountered a failure in the /MongoDB/handshake/success test, which I traced to the lengthened version string (recall that the approach for calculating the current version will now produce something like 1.14.0-20180925+git683f6ac059 for most builds). That string is 30 characters long, while mongoc-handshake-private.h defines HANDSHAKE_DRIVER_VERSION_MAX as 32. This pushes the canary that the unit test looks for almost entirely out of the version string, so much so that the call to strstr() in the test is what fails. So, I increased HANDSHAKE_DRIVER_VERSION_MAX to 48, but then that caused parts of the handshake to not all fit, so then I increased HANDSHAKE_MAX_SIZE, which resulted in this:

error calling ismaster: 'The client metadata document must be less then or equal to 512bytes'

Which takes me back to HANDSHAKE_DRIVER_VERSION_MAX. It appears that I cannot increase it past 32, so from a practical perspective, the test imposes a limit of 20 characters, because the call to strstr() looks for the string "version abc". The question is, do I modify the unit test? Given that the new version scheme produces a 30 character version (31 if the patch version goes to two digits), what single character can I use as a canary? Or, do I modify the version scheme? I am reluctant to remove the Git commit ID, as that is very useful, but Git IDs are not well ordered (which is why I also include the date). In any event, I could find a way to shave some characters off the version, but I would still need something shorter than "version abc" as a canary.

Thoughts? Suggestions?

Comment by A. Jesse Jiryu Davis [ 23/Sep/18 ]

Let’s fall back to 0.0.0 and build successfully with a warning, for convenience. People do want to check out and build from git, outside of MongoDB staff. Include in the warning: instructions for installing GitPython and running the script, or a pointer to where to read those instructions.

Comment by Roberto Sanchez [ 23/Sep/18 ]

jesse, thanks for confirming the versioning scheme.  The main thing I need to know now is whether it is OK to require manual invocation of the version calculation script by the user as an extra step prior to invoking CMake.  This would only be required when building from Git, because the VERSION_CURRENT and VERSION_RELEASED files will no longer be kept under version control. If it is OK to require the invocation of the version calculation script, then the CMake message would change from a warning that the files cannot be found and that we are using version 0.0.0 to a fatal error telling the user to invoke the version calculation script. For us in Evergeen this is not an issue at all since the tasks are themselves scripts and we can invoke arbitrary commands in any order we like, but I am not sure how you would view this as a requirement for users building from Git. If it is not OK to require manual invocation of the version calculation script before CMake, then I will need to come up with a way refresh the variable assignments (the version variables and the variable that stores the name of the distribution tarball) whenever the dist or distcheck targets are invoked. It is not immediately obvious to me how that would work, but I could give it a try.

Comment by A. Jesse Jiryu Davis [ 23/Sep/18 ]

Thanks Roberto. Your version calculation scheme sounds right to me for the C Driver, and it will match how we handle the C++ Driver branches and tags from now on.

Right, it's ok to require Python to make a release archive: we already require Python and Sphinx for the docs portion, it's ok to require Python and GitPython to generate version numbers. As you say, building from a release archive must not require Python.

Comment by Roberto Sanchez [ 23/Sep/18 ]

Assuming what I described in my previous comment is acceptable, the only remaining obstacle that I have to overcome is how to handle invocation of the dist and distcheck targets in the case where VERSION_CURRENT and VERSION_RELEASED are not yet created. I can make those targets depend on the creation of the files (same for the doc targets), but by the point that happens CMake has already failed to find the files and set the version variables to 0.0.0 and determined that the release archive will be called mongo-c-driver-0.0.0.tar.gz.

The "easy" solution is to require that the version calculation script be called prior to invoking CMake. Would that be acceptable? That would be something that would not be a problem in our Evergreen task. Or is it necessary that the version and archive naming be handled completely when CMake and subsequently make are invoked from a Git working directory? (This is not an issue when invoked from within a release archive, as we will ship those files in the archive.)

Comment by Roberto Sanchez [ 23/Sep/18 ]

jesse, kevin.albertson, so I have been looking at the algorithm I implemented for calculating the version number to try to determine if there is a short circuit path to get to the previous release version (what would be stored in VERSION_RELEASED). I have concluded that such a path does not exist in the current implementation of the version calculation script. To further inform my analysis I examined the abi-compliance-check.sh script and found that there is an assumption (perhaps implied) that the version number contained in VERSION_RELEASED will always correspond to a tag in Git. The command git checkout tags/$newest -f in the script supports my conclusion.

Additionally, it is clear that an additional special case must be considered when determining the previous release version: the version may be derived from a tag another branch. For example, when 1.10.0 was released, the previous version was 1.9.5, derived from a tag along the r1.9 branch.

Armed with all of the above I determined what I believe to be the simplest reliable approach that ensures we always find the correct previous version. Simply stated: the previous version is defined as the version represented by the latest git tag matching the regular expression [0-9]+\.[0-9]+\.[0-9]+ and which is strictly a lower version than the version calculated for VERSION_CURRENT. (Note that this specifically includes pre-release tags, as I could no evidence that VERSION_RELEASED has ever contained a pre-release version. It also considers every tag in Git instead of traversing back in history from the current point.)

So, for example, I am currently working on a topic branch and the version calculation for the current version produces the version number 1.11.1-20180922+git67e3e5215e, which would correspond to a previous release version of 1.11.0. For master, the current version is calculated as 1.14.0-20180922+git67e3e5215e, which would correspond to a previous version of 1.13.0, which makes sense. Then, for the r1.13 branch at commit 6afb83ef8, which corresponds to the 1.13.0 tag, the current version calculation is 1.13.0 and the previous version is 1.12.0 (which originates from another branch entirely). This last scenario would be what we would expect for a final release build for 1.13.0.

Based on all of this, the recommended command line flag (I am thinking of -p for previous or prior) would calculate the current version (as is done without the option) and then additionally locate the appropriate tag in the manner I described above. The output of the script would then be the tag that represents the previous version.

Does this approach seem like it will do what we need?

Comment by Roberto Sanchez [ 21/Sep/18 ]

OK. I'll make the necessary modifications.

Comment by Kevin Albertson [ 21/Sep/18 ]

Right, we use it for calculating the tarball URL info in docs in the Sphinx configuration: conf.py
We also use it for the automated ABI checking script: abi-compliance-check.sh

Before adding it to the C driver, could we modify the calc_release_version.py script to also print the most recent release version given some flag (without bumping it)? Both cases could use that python script to determine the most recent release.

Comment by A. Jesse Jiryu Davis [ 21/Sep/18 ]

Great. If that’s all we use VERSION_RELEASED for, then we only use it in a
context where we can rely on Python (when building docs with Sphinx). I
think we can add GitPython to the list of deps for building docs, and
calculate the version while building them.

On Fri, Sep 21, 2018 at 8:35 AM Roberto Sanchez (Jira) <jira@mongodb.org>

Generated at Wed Feb 07 21:16:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.