[CDRIVER-2416] Merge libbson repository into mongo-c-driver Created: 05/Dec/17 Updated: 28/Oct/23 Resolved: 21/May/18 |
|
| Status: | Closed |
| Project: | C Driver |
| Component/s: | Build |
| Affects Version/s: | None |
| Fix Version/s: | 1.10.0 |
| Type: | Epic | Priority: | Major - P3 |
| Reporter: | A. Jesse Jiryu Davis | Assignee: | Roberto Sanchez |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | FY2019Q2 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Description |
|
Currently libbson is in a separate repository from libmongoc; libmongoc includes it as a Git submodule. This causes various problems. It's difficult to build libmongoc with a bundled checkout of libbson using CMake ( Replace the git submodule in libmongoc with the contents of the libbson repository and retire the libbson repository. Do this as a fake author to avoid biasing the GitHub contributor stats. Merge the two test suites into one: no more test-libbson executable, test-libmongoc should now run all the tests for both libraries. Factor the common build logic. If we haven't yet deleted the Autotools scripts ( Continue to ship libbson and libmongoc as distinct libraries; only the repositories are merged. |
| Comments |
| Comment by Roberto Sanchez [ 21/May/18 ] |
|
Woohoo! |
| Comment by A. Jesse Jiryu Davis [ 21/May/18 ] |
|
Yes, go ahead and close it! |
| Comment by Roberto Sanchez [ 21/May/18 ] |
|
jesse, kevin.eric.albertson@gmail.com, are we ready/able to close this Epic now that all of of the constituent tasks are complete? |
| Comment by Githook User [ 15/Apr/18 ] |
|
Author: {'name': 'Roberto C. Sánchez', 'email': 'roberto@connexer.com', 'username': 'rcsanchez97'}Message: |
| Comment by Githook User [ 20/Feb/18 ] |
|
Author: {'email': 'roberto@connexer.com', 'name': 'Roberto C. Sánchez', 'username': 'rcsanchez97'}Message: |
| Comment by Githook User [ 30/Jan/18 ] |
|
Author: {'email': 'jesse@mongodb.com', 'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis'}Message: |
| Comment by Githook User [ 29/Jan/18 ] |
|
Author: {'email': 'jesse@mongodb.com', 'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis'}Message: |
| Comment by Githook User [ 29/Jan/18 ] |
|
Author: {'email': 'jesse@mongodb.com', 'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis'}Message: |
| Comment by Githook User [ 29/Jan/18 ] |
|
Author: {'email': 'jesse@mongodb.com', 'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis'}Message: |
| Comment by Githook User [ 28/Jan/18 ] |
|
Author: {'email': 'jesse@mongodb.com', 'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis'}Message: |
| Comment by Githook User [ 28/Jan/18 ] |
|
Author: {'email': 'jesse@mongodb.com', 'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis'}Message: |
| Comment by Githook User [ 28/Jan/18 ] |
|
Author: {'email': 'jesse@mongodb.com', 'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis'}Message: |
| Comment by Githook User [ 28/Jan/18 ] |
|
Author: {'email': 'jesse@mongodb.com', 'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis'}Message: |
| Comment by Roberto Sanchez [ 16/Jan/18 ] |
|
I have updated the PR with a change that produces a properly versioned libmongoc -> libbson dependency. |
| Comment by Roberto Sanchez [ 12/Jan/18 ] |
|
OK. I will make the change and update the PR. |
| Comment by A. Jesse Jiryu Davis [ 12/Jan/18 ] |
|
It sounds like we should make the dependency versioned, yes. Thanks for catching this. |
| Comment by Roberto Sanchez [ 12/Jan/18 ] |
|
jesse, I wanted to bring to your attention regarding the libbson/libmongoc dependencies. In Debian, libmongoc-1.0-0 depends on libbson-1.0-0, but the dependency is unversioned: https://packages.debian.org/sid/libmongoc-1.0-0 The dependencies of libmongoc-dev -> libbson-dev,libmongoc-1.0-0 and libbson-dev -> libbson-1.0-0 are properly versioned because we explicitly state them in the control file. However, the the dependency libmongoc-1.0-0 -> libbson-1.0-0 is calculated by the dh_shlibdeps helper script. Since we don't do symbol versioning and don't ship a shlibs file, the dependency is unversioned. This isn't necessarily a bad thing, but there is a theoretical possibility that an upgrade could be performed to libmongoc-1.0-0 without a corresponding upgrade to libbson-1.0-0. I think that with the merged repositories it is even less likely that a problem will occur with this because both binary packages will now originate from the same source package. However, it is a bit easier to fix this now because we have a guarantee that both packages will have the same version number because we build from the same source package. We can override the call to dh_makeshlibs during the package build and supply version information to make the dependency versioned. To summarize, if we don't make a change regarding this then the dependencies will continue largely as they have been up to now. The reason I bring this up now is that if we want to change it the approach is more straightforward than it would have been under the split repository setup. |
| Comment by A. Jesse Jiryu Davis [ 09/Jan/18 ] |
|
Sounds great. |
| Comment by Roberto Sanchez [ 09/Jan/18 ] |
|
jesse, I have made what I believe are all the necessary changes. With the rewind recipe you gave me I will be able to simulate a 1.10.0-dev release to build a test package and confirm that everything is in order. My plan is that once I have it tested and I am happy with the Debian packaging I will submit a pull request with the changes. That will allow you to sit on the pull request until you are ready for a 1.10.0 final release. That way, if we need to make a 1.9.1 release we aren't fighting the packaging changes for 1.10.0. |
| Comment by A. Jesse Jiryu Davis [ 09/Jan/18 ] |
|
Hi! I've read your explanation and it seems wise. Please proceed according to your plan. |
| Comment by Roberto Sanchez [ 09/Jan/18 ] |
|
jesse, I have put some thought into this today and I would like to propose a simpler solution than the multiple source tarballs. The bottom line is that if you are OK with the upstream and Debian release tarballs being different today and you are OK with them continuing to be different in the future, then there is no reason we can't just make some minor tweaks to libmongoc Debian packaging and be done with it. Read on for all the gory details. It occurred to me that the current process for generating upstream libbson/libmongoc releases and the process for generating their Debian packages results in two different source tarballs for the same release. The reason for this is that you use some process (maybe 'make dist' or similar) to produce a release tarball and the Debian package process relies the contents of the 'upstream' branch (which is called 'r1.9' in the repository). For example, on master, there is no Makefile or configure script, as the build has to be bootstrapped. The release tarball that you ship includes those items. As a result, if someone downloads your release tarball from GitHub and the .orig.tar.gz for the same upstream release from the Debian package, they would find that the two files had different MD5 sums and different contents. This is not necessarily bad, it just is. In the typical packaging workflow, upstream and Debian packaging develop in separate repositories. The normal workflow has the Debian maintainer import the upstream release tarball into a Debian packaging-specific repository. This is usually done with the assistance of pristine-tar to ensure that the Debian package building process outputs a bit-for-bit identical original tarball as what originated from upstream. However, in our case we share a repository and we just let the Debian package scripts generate a tarball from the point on the upstream branch which is identified in gbp.conf. Since we do not import a tarball with pristine-tar enabled, there is no reference in terms of a specific file to try and match. Personally, I think that the current workflow is good from that standpoint and I think it would be far more trouble than it is worth to try to create a Debian .orig.tar.gz that matches the release tarball you create for upstream release from GitHub. The point of all that preamble is to get to this: since we already produce different release tarballs for upstream releases and Debian packages, we can just create a single libmongoc Debian tarball that contains the sources for both libmongoc and libbson. The whole purpose of multiple upstream tarballs is to allow a Debian source package to be made up of multiple components which are released independently by upstream. However, since the Debian packaging here doesn't use those independent releases and we work on Debian packaging from within a repository that has all the sources together in one place, I think that the right answer is to just migrate the necessary files from the libbson Debian packaging directory into the libmongoc Debian packaging directory. Specifically, the .install, .docs, .lintian-overrides, shlibs, and possibly one or two other. The paths would likely need to be adjusted and I would make sure that the correct build switches and flags get carried through to the libbson build. That would be a whole lot simpler than trying to mess with multiple release tarballs in the packaging process. Trying to do that would require a considerable change to the packaging workflow that we have established and I think that it would actually be a change for the worse. |
| Comment by A. Jesse Jiryu Davis [ 08/Jan/18 ] |
|
Hi Roberto. Yes please do that Debian work next, and please see if there's any shared logic in the two CMakeLists.txt that's worth factoring into shared cmake include files. (Maybe there isn't.) After that, we should factor together the test framework code, but I might be a better person for that work, compared to your skills which are more valuable elsewhere. |
| Comment by Roberto Sanchez [ 08/Jan/18 ] |
|
jesse, I would like to make sure that I have a clear understanding of what I need to accomplish in order to resolve this ticket. Is the intent that I now produce a merged Debian package from the merged repository, or was there something else in addition to or instead of that? |
| Comment by Githook User [ 04/Jan/18 ] |
|
Author: {'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis', 'email': 'jesse@mongodb.com'}Message: Fix libbson's Evergreen test scripts now that its repository is merged |
| Comment by Githook User [ 14/Dec/17 ] |
|
Author: {'name': 'A. Jesse Jiryu Davis', 'email': 'jesse@emptysquare.net', 'username': 'ajdavis'}Message: |
| Comment by Githook User [ 14/Dec/17 ] |
|
Author: {'name': 'A. Jesse Jiryu Davis', 'email': 'jesse@mongodb.com', 'username': 'ajdavis'}Message: libbson's autogen.sh tries to ln -s README.md README, but libbson has |
| Comment by Githook User [ 14/Dec/17 ] |
|
Author: {'name': 'Merge libbson', 'email': 'drivers@mongodb.com'}Message: |
| Comment by Githook User [ 14/Dec/17 ] |
|
Author: {'name': 'A. Jesse Jiryu Davis', 'email': 'jesse@mongodb.com', 'username': 'ajdavis'}Message: |
| Comment by Githook User [ 14/Dec/17 ] |
|
Author: {'name': 'A. Jesse Jiryu Davis', 'email': 'jesse@mongodb.com', 'username': 'ajdavis'}Message: |
| Comment by A. Jesse Jiryu Davis [ 05/Dec/17 ] |
|
From Roberto:
|