[CXX-1158] Unable to link against SSL libs on MacOS Created: 08/Dec/16  Updated: 20/Dec/16  Resolved: 20/Dec/16

Status: Closed
Project: C++ Driver
Component/s: Build
Affects Version/s: 3.1.0-rc0
Fix Version/s: 3.2.0-rc0

Type: Bug Priority: Major - P3
Reporter: Jonathan Wang Assignee: David Golden
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

macOS 10.12.1


Attachments: Text File 0001-CXX-1158-Add-OS-X-framework-support-to-FindLibMongoC.patch     Text File pkg_flags_darwin.txt     Text File pkg_flags_openssl.txt     Text File ssl-darwin-output.txt     Text File ssl-openssl-output.txt    
Issue Links:
Related
is related to CDRIVER-1058 Install CMake 'config' style files fo... Closed

 Description   

The build for the Mongo C++ library is unable to link to the SSL libraries when the Mongo C driver is configured and built using `--ssl=auto/darwin`.

Previously, with the Mongo C++ driver at version 3.0.3, the CMake configuration was able to locate the Mongo C library and its dependencies (OpenSSL libraries) using pkg_config.

See the attached file for the commands and the output.



 Comments   
Comment by Githook User [ 20/Dec/16 ]

Author:

{u'username': u'xdg', u'name': u'David Golden', u'email': u'xdg@xdg.me'}

Message: CXX-1170 Add OS X framework support to FindLibMongoC.cmake

Backported from CXX-1158
Branch: bugfix/3.1.x
https://github.com/mongodb/mongo-cxx-driver/commit/fcce96df6baa3369374dcb4f46d198c585216204

Comment by Githook User [ 20/Dec/16 ]

Author:

{u'username': u'xdg', u'name': u'David Golden', u'email': u'xdg@xdg.me'}

Message: CXX-1158 Add OS X framework support to FindLibMongoC.cmake
Branch: master
https://github.com/mongodb/mongo-cxx-driver/commit/326357a91e6267b9ce3a7ca5904624aec4b80443

Comment by A. Jesse Jiryu Davis [ 16/Dec/16 ]

Acknowledged acm, CDRIVER-1058 is now top-of-backlog.

Comment by David Golden [ 15/Dec/16 ]

Excellent! We'll get this merged in for a 3.1.1 release probably next week.

Comment by Jonathan Wang [ 15/Dec/16 ]

I can confirm that this patch resolves the issue for me. It allows the C++ library to correctly link with Darwin SSL libraries/frameworks, and when using OpenSSL, the build continues to work as it did before.

Comment by Andrew Morrow (Inactive) [ 15/Dec/16 ]

I would support the C driver emitting those as well, since currently the lack of them means building the C++11 driver is difficult on Windows, where there is usually not pkg-config. I have previously filed this, as CDRIVER-1058.

Comment by Jonathan Wang [ 15/Dec/16 ]

I also agree that having libmongoc generate a .cmake file will probably help with some of these CMake configuration issues, so that might be a good way to go in the future. The CMake mechanism for package dependency gives you more control over exactly how the variables are set, so you wouldn't have to deal with any strange pkg-config issues where certain flags end up in the wrong variables.

I'll try out the patch and also see if I can think of any alternative ways to do it with pkg-config.

Comment by David Golden [ 15/Dec/16 ]

Possibly, having libmongoc generate a .cmake file might help, as the CMake mechanism for package dependency detection using those .cmake file (versus the pkg-config) supposedly (according to stack overflow) does the right thing with frameworks. But I'm not sure how much I'd bet on that being true.

Comment by A. Jesse Jiryu Davis [ 15/Dec/16 ]

Very curious to see if there's a better way, LMK if you get a reply there.

Comment by David Golden [ 15/Dec/16 ]

FWIW, I also raised this issue on the CMake mailing list: https://cmake.org/pipermail/cmake/2016-December/064711.html

Comment by David Golden [ 15/Dec/16 ]

jwang, I've attached a patch to this ticket. Could you please try it out and see if it fixes the problem you're having? It works on my side, but would like a double check before I commit it.

Comment by David Golden [ 14/Dec/16 ]

I'm able to replicate this now. I've moved this ticket to "Open" and am scheduling it for 3.1.1.

Comment by Jonathan Wang [ 14/Dec/16 ]

I deleted the old build output files and replaced them with new ones that I ran just now. These new files should correctly show the commands and outputs.

Comment by Jonathan Wang [ 14/Dec/16 ]

I think I may have copied the wrong output into the file, and that's why it shows that SSL is configured for "darwin" instead of "openssl". I'll try to regenerate those files for you.

Comment by David Golden [ 14/Dec/16 ]

For one of the commands, ssl=darwin is used. For the other command, ssl=openssl is used.

I understand, but I'd like to see the exact actual command you used.

The ssl-darwin-output.txt file you uploaded has this:

Mongo C library
===============
cd mongo-c-driver
PKG_CONFIG_LIBDIR=/usr/local/custom/openssl/lib/pkgconfig ./autogen.sh --prefix=/usr/local/custom/mongodb/ \
    --disable-shared --enable-static --enable-rdtscp --disable-sasl --enable-ssl=openssl --with-libbson=bundled \
    --disable-automatic-init-and-cleanup
make -j4
sudo make install

That obviously isn't the actual command you used, as it has "--enable-ssl=openssl". I'd like the see what you actually used, including any PKG_CONFIG_LIBDIR you set, the other options set, etc, as well as the configure output.

Likewise, your ssl-openssl-output.txt has this for configuration output:

Build configuration:
  Enable debugging (slow)                          : no
  Compile with debug symbols (slow)                : no
  Enable GCC build optimization                    : yes
  Enable automatic init and cleanup                : yes
      DEPRECATED: use --disable-automatic-init-and-cleanup
  Code coverage support                            : no
  Cross Compiling                                  : no
  Fast counters                                    : yes
  Shared memory performance counters               : yes
  SASL                                             : no
  SSL                                              : darwin
  Libbson                                          : bundled

That says that SSL is configured for "darwin", not openssl.

Would you mind re-generating those files being sure to get the correct command lines with the correct output? That would help me be sure I'm replicating your situation exactly.

Thank you!

Comment by Jonathan Wang [ 14/Dec/16 ]

This problem is indeed only shows up for static builds and not dynamic builds.

Comment by Jonathan Wang [ 14/Dec/16 ]

For one of the commands, ssl=darwin is used. For the other command, ssl=openssl is used.

Comment by David Golden [ 14/Dec/16 ]

Additional note: I can't replicate this when compiling for dynamic linking, so this appears specific to static builds.

Comment by David Golden [ 14/Dec/16 ]

Jonathan, I'm confused by your sample output. When I diff them, there's no difference in the commands you ran, but differences in pkg-config output and link output. See this gist.

Comment by David Golden [ 14/Dec/16 ]

Thanks for all the analysis.
As a heads up, even once the linking is resolved, SSL support is currently broken in 3.1.0 due to a bizarre rogue merge. It's fixed in CXX-1163 and we expect to ship a patch release by next week. We'll try to get the linking sorted out at the same time.

Comment by Jonathan Wang [ 12/Dec/16 ]

From my own experience in CMake with linking OS X frameworks and from what I've seen other people do, usually the standard CMake approach with these frameworks is to use find_library like so:

find_library(FRAMEWORK_PATH ${FRAMEWORK_NAME})
target_link_libraries(target ${FRAMEWORK_PATH})

example:

find_library(FOUNDATION_FW Foundation)
target_link_libraries(mongocxx ${FOUNDATION_FW})

The results from the find_library function are full paths to the frameworks, and that's probably why it's preferred over providing the -framework Foundation linker flag, which seems to be more tricky in getting it to work.

Comment by Jonathan Wang [ 12/Dec/16 ]

One way to resolve the issue of the -framework Security -framework CoreFoundation flags not showing up in the linking stage is to use LIBMONGOC_LDFLAGS instead of LIBMONGOC_LIBRARIES for the mongocxx_libs used for the target_link_libraries (making this change isn't sufficient as a permanent fix for several reasons, but it can be done to help debug this current issue).

However, upon providing LIBMONGOC_LDFLAGS to target_link_libraries, the verbose link command now shows -lmongoc-1.0 -framework -lSecurity -framework -lCoreFoundation -lbson-1.0. Note the -l in front of the Security and CoreFoundation components, which is not the intended behavior.

Comment by Jonathan Wang [ 12/Dec/16 ]

I can also confirm (from looking at the CMakeLists.txt inside src/mongocxx) that these values are provided to the linking step by the LIBMONGOC_LIBRARY_DIRS and LIBMONGOC_LIBRARIES variables. They show up in the CMakeCache.txt file as well.

For the Darwin SSL version of MongoDB C:

LIBMONGOC_LIBRARIES:INTERNAL=mongoc-1.0;bson-1.0
LIBMONGOC_LIBRARY_DIRS:INTERNAL=/usr/local/custom/mongodb/lib

For the OpenSSL version of MongoDB C++ driver build:

LIBMONGOC_LIBRARIES:INTERNAL=mongoc-1.0;ssl;crypto;bson-1.0
LIBMONGOC_LIBRARY_DIRS:INTERNAL=/usr/local/custom/mongodb/lib;/usr/local/custom/openssl/lib;/usr/local/custom/mongodb/lib

You can see that the CMake variables above do not contain the -framework flags necessary to link to the libraries needed for SSL. I suspect this has something to do with the way the CMake PkgConfig module works. I've also attached files containing snippets of text from the CMakeCache.txt file for both the Darwin and OpenSSL builds.

Comment by Jonathan Wang [ 12/Dec/16 ]

I've attached two new files (and removed the old file) showing the commands I'm using and the output/error that is generated for when I install the Mongo C/C++ driver with --ssl=darwin and --ssl=openssl.

If you look at the output for the Mongo C++ library build step (when built against the Mongo C driver with darwin SSL), when it attempts to link libmongocxx.dylib, it fails with an error as previously mentioned. In the build command, it shows -lmongoc-1.0 -lssl -lcrypto -lbson-1.0 -lbson-1.0 even though the pkg-config --cflags --libs libmongoc-1.0 shows that the libraries should be -lmongoc-1.0 -framework Security -framework CoreFoundation -lbson-1.0.

It seems as though some part of the Mongo C++ driver might not be correctly setup to get the libraries from pkg-config. I'll continue to investigate to see if I can figure out anymore information.

Comment by A. Jesse Jiryu Davis [ 09/Dec/16 ]

FYI, when I build libmongoc with --enable-ssl=darwin, it seems to give the right libraries:

> pkg-config --cflags --libs libmongoc-1.0
-I/usr/local/include/libmongoc-1.0 -I/usr/local/include/libbson-1.0 -L/usr/local/lib -lmongoc-1.0 -lsasl2 -framework Security -framework CoreFoundation -lbson-1.0

Comment by David Golden [ 09/Dec/16 ]

Thanks for the report. We'll try to reproduce this and get back to you.

Generated at Wed Feb 07 22:01:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.