[CDRIVER-2083] Distribute CMake config packages for statically linking to libbson and libmongoc Created: 09/Mar/17  Updated: 15/Jun/17  Resolved: 12/Jun/17

Status: Closed
Project: C Driver
Component/s: Build, cmake, libbson, libmongoc
Affects Version/s: None
Fix Version/s: 1.7.0

Type: New Feature Priority: Major - P3
Reporter: A. Jesse Jiryu Davis Assignee: Hannes Magnusson
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by CXX-1257 Revise static library naming and link... Closed
Related
related to CDRIVER-2166 Improve autotools/pkg-config detection Closed
related to CDRIVER-2175 libbson cmake: pthreads not correctly... Closed
is related to CDRIVER-2157 Linking to libmongoc with MinGW-64 fa... Closed
is related to CXX-1330 Remove RHEL 5.5 Evergreen builder Closed
Epic Link: convenient-static

 Comments   
Comment by Githook User [ 15/Jun/17 ]

Author:

{u'username': u'bjori', u'name': u'Hannes Magnusson', u'email': u'bjori@php.net'}

Message: CDRIVER-2083 Fix windows build
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/206fcfb774154b24657c379b9f2af7704479500b

Comment by Hannes Magnusson [ 12/Jun/17 ]

We consider this ticket to be fixed now.

The Cmake system is experimental and not intended for production use for platforms other then Windows.

That being said, we do distribute now

  • libmongoc-1.0-config-version.cmake
  • libmongoc-1.0-config.cmake
  • libmongoc-1.0-static-config-version.cmake
  • libmongoc-1.0-static-config.cmake
  • libbson-1.0-config-version.cmake
  • libbson-1.0-config.cmake
  • libbson-1.0-static-config-version.cmake
  • libbson-1.0-static-config.cmake

These can be used in native CMake projects via find_package ().
This support is experimental.

Using the PkgConfig module, even with CMake builds, remains our production recommendation, also on Windows.

The mongoc CMakeLists.txt still needs work to get up to par with the autotools build system.
The libbson one probably too. This will be addressed hopefully in future versions, see CDRIVER-2145

Comment by Githook User [ 12/Jun/17 ]

Author:

{u'username': u'bjori', u'name': u'Hannes Magnusson', u'email': u'bjori@php.net'}

Message: CDRIVER-2083 Use consistent path for the artifacts
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/6ffc637e9225e6080cd5a44e9e7e2afe4b6a5ab8

Comment by Githook User [ 12/Jun/17 ]

Author:

{u'username': u'bjori', u'name': u'Hannes Magnusson', u'email': u'bjori@php.net'}

Message: CDRIVER-2083 Raise a warning when using CMake on non-Windows platform
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/6040421156526930a8d4adc9acd222696454137c

Comment by Githook User [ 12/Jun/17 ]

Author:

{'username': 'bjori', 'name': 'Hannes Magnusson', 'email': 'bjori@php.net'}

Message: CDRIVER-2083 Fix hardcoded dependencies
Branch: master
https://github.com/mongodb/libbson/commit/7bb7b5759450230ea85b5b3fe2134b5259438050

Comment by Githook User [ 12/Jun/17 ]

Author:

{'username': 'bjori', 'name': 'Hannes Magnusson', 'email': 'bjori@php.net'}

Message: CDRIVER-2083 Raise a warning when using CMake on non-Windows platform
Branch: master
https://github.com/mongodb/libbson/commit/c56da2b27e6a630500f19ea440906d1615d3edbc

Comment by J Rassi [ 28/Apr/17 ]

One more issue: as of current libbson master, I get the following error when statically linking the bsoncxx tests against libbson on the rhel55-test builders (but not the other C++ driver Evergreen builders):

 [2017/04/28 03:03:00.632] /data/mci/ea27da35ca110e44b168a5444e84c702/mongoc/lib/libbson-static-1.0.a(libbson_la-bson-clock.o): In function `bson_get_monotonic_time':
 [2017/04/28 03:03:00.632] /data/mci/ea27da35ca110e44b168a5444e84c702/mongo-cxx-driver/mongo-c-driver-1.7.0-dev/src/libbson/src/bson/bson-clock.c:124: undefined reference to `clock_gettime'
 [2017/04/28 03:03:00.632] collect2: error: ld returned 1 exit status
 [2017/04/28 03:03:00.632] make[2]: *** [src/bsoncxx/test/test_bson] Error 1

It seems like explicitly adding -lrt to BSON_STATIC_LIBRARIES is required on certain platforms, when clock_gettime() is available and being used.

Comment by Githook User [ 28/Apr/17 ]

Author:

{u'username': u'xdg', u'name': u'David Golden', u'email': u'xdg@xdg.me'}

Message: CDRIVER-2083 Split absolute paths used in .pc files

When generating .pc files from CMake, we don't want to put absolute
library paths in the .pc files (which is how CMake prefers them) because
pkg-config will re-order items without '-l' options.

This commit splits absolute paths back into separate -L and -l
components. E.g. `/opt/foo/lib/libbar.a` becomes `-L/opt/foo/lib` and
`-lbar`.
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/04707295cc89d0d9b875da991743a8f8e07cc42e

Comment by Githook User [ 28/Apr/17 ]

Author:

{u'username': u'xdg', u'name': u'David Golden', u'email': u'xdg@xdg.me'}

Message: CDRIVER-2083 Fix CMake conditions for WIN32 not MSVC
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/4327e32116a5bb9675ed9a812dc9765276f6d6ac

Comment by Githook User [ 28/Apr/17 ]

Author:

{'name': 'David Golden', 'email': 'xdg@xdg.me', 'username': 'xdg'}

Message: CDRIVER-2083 Fix Windows/Unix CMake conditionals

Fixes two cases that were conditional on MSVC and shouldn't have been:

  • Uses WIN32 for requiring ws2_32 or not
  • Uses UNIX for configuring Threads or not

The latter fix just moves `find_package(Threads)` next to commands
in a later "if UNIX" stanza that uses CMAKE_THREADS_LIBS_INIT.
Branch: master
https://github.com/mongodb/libbson/commit/f01e0fe9bfdbc62fe5c87e01a2c9935a26bd2b31

Comment by Githook User [ 28/Apr/17 ]

Author:

{'name': 'David Golden', 'email': 'xdg@xdg.me', 'username': 'xdg'}

Message: CDRIVER-2083 use WIN32 not MSVC conditional in .cmake
Branch: master
https://github.com/mongodb/libbson/commit/fd190bc0598cb45eeccd6e7801ba526dd7294c5e

Comment by Githook User [ 27/Apr/17 ]

Author:

{u'username': u'xdg', u'name': u'David Golden', u'email': u'xdg@xdg.me'}

Message: CDRIVER-2083 Add MONGOC_STATIC to static .pc file
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/20b08ec5711ceb555c57b90bcb4351373ea4d9ab

Comment by A. Jesse Jiryu Davis [ 26/Apr/17 ]

I don't mind leaving the message calls, or a subset of them, in the documentation. They may actually be informative to readers, and encourage them to use diagnostic messages in their own CMakeLists when they're trying to link to libbson or libmongoc.

Comment by David Golden [ 26/Apr/17 ]

I've taken this ticket, as there's some additional cleanup work to do once we see how the updated code smokes in the CI system:

  • Rationalize PUBLIC/PRIVATE/INTERFACE usage in CMakeList.txt for use with test and example targets
  • Possibly convert examples in CMakeList.txt to use libmongoc as a standalone program would instead of building using the already defined mongoc modules
  • Remove diagnostic debugging message calls from examples/cmake/* code so it doesn't appear in generated documentation
Comment by Githook User [ 26/Apr/17 ]

Author:

{u'username': u'xdg', u'name': u'David Golden', u'email': u'xdg@xdg.me'}

Message: CDRIVER-2083 Update .cmake variable naming scheme

This commit updates the libbson submodule commit point.

libbson already publishes two .cmake files, one for shared builds and one
for static builds. The updated libbson commit publishes variables under
separate namespaces, BSON_* and BSON_STATIC_*.

This commit uses those .cmake files directly instead of our own FindBSON
config file and uses the new variable names where appropriate.

This commit also changes the generated .cmake files for libmongoc to
follow the same MONGOC_* and MONGOC_STATIC_* naming convention for
downstream users.

mongocxx (and maybe the PHP driver) will need a corresponding update.
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/5bb1e4b61de99a432cd39111c841c29ac0cc621d

Comment by Githook User [ 26/Apr/17 ]

Author:

{'name': 'David Golden', 'username': 'xdg', 'email': 'xdg@xdg.me'}

Message: CDRIVER-2083 Revise .cmake variable names for consistency
Branch: master
https://github.com/mongodb/libbson/commit/399288e272c66afbada2a0647e33a381bd1030d0

Comment by Githook User [ 21/Apr/17 ]

Author:

{u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}

Message: CDRIVER-2083 fix semicolons in pkg-config files

Same as the CMake fix, now with Autools.
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/724074e581241634feb1ad563db1a66b25cd630e

Comment by Githook User [ 21/Apr/17 ]

Author:

{u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}

Message: CDRIVER-2083 fix linking test
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/7cce898003aa6d5065eda00a40c4ed3a2c52a823

Comment by Githook User [ 21/Apr/17 ]

Author:

{'name': 'David Golden', 'username': 'xdg', 'email': 'xdg@xdg.me'}

Message: CDRIVER-2083 use absolute lib paths in CMake config files

This commit changes .cmake files to return libraries with absolute paths,
which removes the need for CMake end-users to set `link_directories`.

See https://cmake.org/cmake/help/v3.0/command/link_directories.html
for additional context.
Branch: master
https://github.com/mongodb/libbson/commit/a1131cccedb2e0f4447194886febdfa025aa03f5

Comment by Githook User [ 21/Apr/17 ]

Author:

{'name': 'David Golden', 'username': 'xdg', 'email': 'xdg@xdg.me'}

Message: CDRIVER-2083 add target_compile_definitions to shared library example
Branch: master
https://github.com/mongodb/libbson/commit/bc655d8b1fc93cc947f82a19ad50f5c8a57ae079

Comment by Githook User [ 21/Apr/17 ]

Author:

{'name': 'David Golden', 'username': 'xdg', 'email': 'xdg@xdg.me'}

Message: CDRIVER-2083 use target_compile_definitions

add_definitions() has global effect, so isn't good to include
in a .cmake file. This commit instead publishes the definitions in a
variable and lets consumers set it on a target the same way they set
library/include paths.

libmongoc will need a corresponding update.
Branch: master
https://github.com/mongodb/libbson/commit/18da877a7b25ce5caa87d256c2d8b734ac813342

Comment by Githook User [ 21/Apr/17 ]

Author:

{u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}

Message: CDRIVER-2083 fix semicolons in pkg-config files

When libmongoc-1.0.pc and libmongoc-static-1.0.pc were generated by
CMake, the "Libs" list could be semicolon-delimited, rather than
space-delimited.
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/11cda664a795659c7085f2ded7064807d737bb2e

Comment by Githook User [ 21/Apr/17 ]

Author:

{'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis', 'email': 'jesse@mongodb.com'}

Message: CDRIVER-2083 set BSON_STATIC_LIBRARIES for CMake

When a CMake project links to libbson statically it should do:

find_package (libbson-static-1.0)

With this change, the above command now sets BSON_STATIC_LIBRARIES, not
BSON_LIBRARIES, so a CMake project can now link some of its targets to
libbson statically and some dynamically.

Also, define BSON_STATIC when building the libbson static library. This
avoids marking libbson's public functions for dynamic-library export
when building libbson's static lib. It probably has no effect either
way but seems like the logical thing to do.
Branch: master
https://github.com/mongodb/libbson/commit/4ece6d16b196a7e4df943b88615889cd7fcea876

Comment by A. Jesse Jiryu Davis [ 20/Apr/17 ]

rassi you commented April 12. You said that "the libmongoc shared library may be falsely advertising that it exports libbson API functions" because it defines BSON_COMPILATION. That's fixed now, there's a separate MONGOC_COMPILATION macro.

"The mongoc_static target (and nearly all libmongoc targets) depends on bson-1.0, not bson-1.0-static." I'll fix that. First, libbson-static-1.0-config.cmake.in should set a different variable, BSON_STATIC_LIBRARIES, instead of BSON_LIBRARIES. The other variables BSON_INCLUDE_DIRS and BSON_LIBRARY_DIRS will always be the same whether you use libbson statically or dynamically.

Then libmongoc should run:

find_package (libbson-1.0)
find_package (libbson-static-1.0)

Then it will build its own static lib by statically linking to libbson's static lib, while continuing to build its dynamic lib by dynamically linking to libbson's dynamic lib.

You also wrote, "The libbson and libmongoc static libraries are marking their API functions for DLL export." I don't think that matters, since when a static library is compiled (with any of our compilers), marking a function for DLL export has no effect.

david.golden you wrote, "The C++ driver has worked around these issues by only building static or dynamic and not both." Since our Autotools build system can generate both at once, it must able to generate CMake config files that handle the situation where both static and dynamic libbson libs are install, or both static and dynamic libmongoc libs. I want to generate the same config files when libbson or libmongoc are built and installed with CMake, so the CMake config files generated by CMake must also handle the situation where both static and dynamic libs are installed. Therefore, when you build libbson and libmongoc with CMake, the simplest thing is to always build both static and dynamic libs, and generate CMake config files that can handle the situation where both static and dynamic libs are installed, by defining distinct variables like BSON_LIBRARIES and BSON_STATIC_LIBRARIES.

Comment by Githook User [ 18/Apr/17 ]

Author:

{u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}

Message: CDRIVER-2083 libmongoc-static-1.0-config.cmake sets MONGOC_STATIC

Also test static and dynamic linking, with and without SSL, on Windows.
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/9ed167d84c80ddad4da2fc2adb32a7808521b139

Comment by J Rassi [ 18/Apr/17 ]

There is a variable substitution issue in the CMake-generated pkg-config files that affects libmongoc master.

My installed libmongoc-1.0.pc file looks like the following (Mac OS X). Note the invalid semicolon in the "Libs" section:

prefix=/opt/static_test/libmongoc
exec_prefix=${prefix}
libdir=${prefix}/lib
includedir=${exec_prefix}/include
 
Name: libmongoc
Description: The libmongoc MongoDB client library.
Version: 1.7.0-dev
Requires: libbson-1.0
Libs: -L${libdir} -lmongoc-1.0 /usr/lib/libsasl2.dylib /usr/lib/libssl.dylib;/usr/lib/libcrypto.dylib
Cflags: -I${includedir}/libmongoc-1.0

I can reproduce the issue as follows, from the root of the mongo-c-driver repository on my Mac, against libmongoc master (5857491):

cmake -DCMAKE_PREFIX_PATH=/path/to/libbson -DCMAKE_INSTALL_PREFIX=/path/to/libmongoc . && make -j32 && make install

My local temporary fix for this issue is to introduce the following code into CMakeLists.txt, and then to replace @SSL_LIBS@ in the relevant pc.in files with @SSL_LIBS_COPY@:

foreach(FLAG ${SSL_LIBS})
    set(SSL_LIBS_COPY "${SSL_LIBS_COPY} ${FLAG}")
endforeach()

Note that 1.6.2 is not affected (these libraries are omitted from the "Libs" section in libmongoc-1.0.pc on 1.6.2), and the autotools-generated pkg-config files on master are not affected (in those files, these libraries are listed as "-lssl -lcrypto").

Comment by Githook User [ 17/Apr/17 ]

Author:

{'email': 'jesse@mongodb.com', 'username': 'ajdavis', 'name': 'A. Jesse Jiryu Davis'}

Message: CDRIVER-2083 add ws2_32 to windows static cmake config

Also test linking with cmake config files on Windows in Evergreen.
Branch: master
https://github.com/mongodb/libbson/commit/b4b9c8bfe4d481cdfba14721665c0bd4cd12110a

Comment by David Golden [ 14/Apr/17 ]

libmongoc PR is PR/187

Comment by David Golden [ 14/Apr/17 ]

The C++ driver has worked around these issues by only building static or dynamic and not both, as controlled by the BUILD_SHARED_LIBS variable. While CMake doesn't exactly have a "standard" way to do things, this approach, toggling static or shared, seems often recommended and is used by large, well known libraries like googletest.

In the long run, the C driver could explore making a similar change, but in the short run, not changes are necessary to unblock the C++ driver.

However, I do believe that libbson should not globally add -DBSON_STATIC to definitions but should instead publish it to a variable for consumers to use in a targeted way. This is implemented in PR/187.

I think this PR (and a corresponding change in libmongoc) should be part of the 1.7.0 release.

Comment by J Rassi [ 12/Apr/17 ]

The CMake config files added in the commits above suffer from the issue that the static and non-static versions both export variables with the same names. We should either change them to export variables with different names, or implement the decision that projects relying on them cannot support linking against both the static and non-static versions of the libraries in a single build.

Currently, the mongoc_static target (and nearly all libmongoc targets) depends on bson-1.0, not bson-1.0-static. This is due to the fact that libmongoc only calls find_package() on bson-1.0; it doesn't know about the existence of the bson-static-1.0 library at all. This causes some unintuitive results: for example, the mongoc tests (e.g. "test-libmongoc") are linking against static mongoc, but dynamic libbson.

I've also discovered the following additional potential issues with the export macros, as written. Consider these issues to be minor, since I don't yet have evidence that they have user impact.

  • The libmongoc shared library may be falsely advertising that it exports libbson API functions. This is due to the fact that defining BSON_COMPILATION causes BSON_API to expand to __declspec(dllexport), and mongoc_shared is defining BSON_COMPILATION when including libbson header files. My suggested fix for this issue is to replace the "#elif defined(BSON_COMPILATION)" condition in bson-macros.h with some other "#elif defined(X)" condition, where X is a macro that is defined during libbson compilation but not defined during libmongoc compilation.
  • The libbson and libmongoc static libraries are marking their API functions for DLL export. This is because neither of them define their X_STATIC macros during compilation. My suggested fix for this issue is to define MONGOC_STATIC when compiling libmongoc, and to define BSON_STATIC when compiling both. This is consistent with advice given in the CMake manual ("If the same sources are used to create both a shared and a static library, the uppercased symbol ${BASE_NAME}_STATIC_DEFINE should be used when building the static library", from https://cmake.org/cmake/help/v3.2/module/GenerateExportHeader.html).
Comment by A. Jesse Jiryu Davis [ 07/Apr/17 ]

One more task before this is done: Test static and dynamic linking on Windows in Evergreen for libbson and libmongoc. So far I've only done Linux.

Comment by Hannes Magnusson [ 28/Mar/17 ]

Always distributing the cmake package file sounds like a bad idea to me.

The cmake manifest is still very much an afterthought and doesn't support nearly everything it should, not even building against the bundled libbson.

Installing the cmake package helpers makes this support though become official and irremovable, which means we now have to spent effort into fixing the platform incompatibilities it has, other then just using it for generating windows manifests

Comment by Githook User [ 27/Mar/17 ]

Author:

{u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}

Message: CDRIVER-2037, CDRIVER-2083 convenient linking

Whether built with CMake or the Autotools, libmongoc now installs CMake
config-file packages for libmongoc-1.0 and libmongoc-static-1.0, and pkg-config
files for the same.

Compile and link an example program with all combinations: libmongoc built with
CMake or the Autotools, using static or dynamic linking, and finding libmongoc
with the CMake find_package command or with pkg-config.
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/3a522396807209076e1cbee96da6a604112a4243

Comment by Githook User [ 20/Mar/17 ]

Author:

{u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}

Message: CDRIVER-2083 add semicolon in Makefile.am
Branch: master
https://github.com/mongodb/libbson/commit/473407a9b8b55b8ab09b2e0d7d89177c8ca01a5a

Comment by Githook User [ 20/Mar/17 ]

Author:

{u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}

Message: CDRIVER-2083 allow find_package with no version
Branch: master
https://github.com/mongodb/libbson/commit/1e613c716c13d7e6468d559a95d3e6938db02cba

Comment by Githook User [ 20/Mar/17 ]

Author:

{u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}

Message: CDRIVER-2083 fix cmake path in test script
Branch: master
https://github.com/mongodb/libbson/commit/cfc24f9eaddd0db88a834d69cfdee642b1b5ffd5

Comment by Githook User [ 20/Mar/17 ]

Author:

{u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}

Message: CDRIVER-2037, CDRIVER-2083 easy linking pt 2

Add and test config-version.cmake files, update example CMake style,
more cleanly test in Autotools install-exec-hook whether static linking
is enabled, clearer test matrix.
Branch: master
https://github.com/mongodb/libbson/commit/e50bfae13a7ca2042be03fabf577c76815800773

Comment by A. Jesse Jiryu Davis [ 15/Mar/17 ]

Done for libbson. Still to do for libmongoc.

Comment by Githook User [ 15/Mar/17 ]

Author:

{u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}

Message: CDRIVER-2037, CDRIVER-2083 easy static linking

Distribute separate pkg-config files and CMake config packages for
programs that want to statically link to libbson.
Branch: master
https://github.com/mongodb/libbson/commit/66baac8c6cb228a3a5e2af3d90442c868a2bb54e

Generated at Wed Feb 07 21:14:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.