[CXX-1489] CMake can't find pkgconfig when configuring the C++ driver Created: 04/Jan/18 Updated: 27/Oct/23 Resolved: 17/Jan/18 |
|
| Status: | Closed |
| Project: | C++ Driver |
| Component/s: | Build |
| Affects Version/s: | 3.1.2 |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Adwait Lele | Assignee: | Unassigned |
| Resolution: | Works as Designed | Votes: | 0 |
| Labels: | build, install | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
CentOS 7.4 on VMWare ESXi with MongoDB 3.4 installed |
||
| Description |
|
I'm using the steps of this site (https://mongodb.github.io/mongo-cxx-driver/mongocxx-v3/installation/) and I'm CMake is throwing this error: A required package was not found I have "pkgconfig" installed, not "pkg-config". Is that the issue? It seems that "pkg-config", like stated in the steps from the link above, only exists for Debian/Ubuntu while for CentOS it's called "pkgconfig" |
| Comments |
| Comment by Andrew Morrow (Inactive) [ 17/Jan/18 ] | |||||||||||||||||||||||||||||||
|
Happy to hear you got it working. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 17/Jan/18 ] | |||||||||||||||||||||||||||||||
|
That worked! Finally. With the c++/g++ compile method (no CMake or pkg-config) adding the linked argument worked and it wrote to a DB. It didn't work with the pkg-config version though, but I don't think I'll need that anyways. Thanks for being patient. I think this issue can be closed now. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 17/Jan/18 ] | |||||||||||||||||||||||||||||||
|
Yeah the README says that the 3.0.x series is recommended for all future development, so it's confusing. Also git describe --all works, not -a, just a note. It returns tags/r3.1.3 so I guess it is 3.1.x series so I should be good for MongoDB 3.4. Let me try the rpath linker argument thing^ you mentioned. | |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 17/Jan/18 ] | |||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 16/Jan/18 ] | |||||||||||||||||||||||||||||||
|
Also, how can I tell what version I have installed. I assumed it installed 3.1.x when I ran:
But now I am not sure if it installed 3.0.x. I have MongoDB 3.4, so I need to have 3.1.x installed, otherwise it won't work. I can't see anywhere in the mongo-cxx-driver folder what version it is either. The README says 3.0.x series is recommended for all new development so I am assuming I have the older one unfortunately, but it doesn't outright say it. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 16/Jan/18 ] | |||||||||||||||||||||||||||||||
|
It doesn't recognize -rpath option, when I "-rpath /usr/local/lib to that second compile method call^:
| |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 16/Jan/18 ] | |||||||||||||||||||||||||||||||
|
The -L argument sets the linker search path at link time. The -rpath argument sets the library search path at runtime. You need both. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 16/Jan/18 ] | |||||||||||||||||||||||||||||||
|
Did you mean the lib directory where the libmongocxx.so._noabi and libsoncxx.so._noabi are located? I see them that in /usr/local/lib. That path seems to be included in that last line of the second compilation method I tried
| |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 16/Jan/18 ] | |||||||||||||||||||||||||||||||
|
Would the top answer here (https://stackoverflow.com/questions/33050113/how-to-install-boost-devel-1-59-in-centos7) be something that might work for "correctly" donwloading Boost? The issue is, I don't know who wrote the rpm and if it's credible. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 16/Jan/18 ] | |||||||||||||||||||||||||||||||
|
Here are the 2 ways that I compiled my code (test.cpp): Compiling with the help of pkg-config
Compiling without pkg-config or CMake
What do you mean the lib-dir-of-libmongocxx? There is no "lib" directory in mongo-cxx-driver or in libmongoc-1.0? On the Boost header aspect - the only capability I need from mongo-cxx-driver is to be able to write JSON (BSON) to a MongoDB database through a C++ application I already have. Right now I am printing the values to the console, but I'd like to put them in a JSON and write them to a DB, which I am going to query using Node/Angular. I don't think the way I "installed" Boost should be an issue, would it? | |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 16/Jan/18 ] | |||||||||||||||||||||||||||||||
|
Did you build your application such that the linker runpath was configured to include the installation location for the C++ driver libraries? If not, it won't be able to find them. You probably need to build with -Wl,rpath,<path-to-lib-dir-of-libmongocxx> on your application link line. Also, replacing system headers like that is very dangerous. It will work for header only boost libraries, but if you try to use anything from boost that needs to link to the boost libraries for symbol definitions, it isn't going to work. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 16/Jan/18 ] | |||||||||||||||||||||||||||||||
|
Since I was having more success with the 3.1.2 version I just built that. It said I had an old version of boost (1.53 is too old) by looking at the boost folder in /usr/include. So I downloaded the headers for boost 1.66 and replaced them in /usr/local/boost. The CMake ran with the boost option and without checking for pthread. Also make && sudo make install ran without any issues. Didn't get stuck at 81% like before and it seemed to build correctly. The issue I'm having now is actually testing the driver out. Step 6 from (https://mongodb.github.io/mongo-cxx-driver/mongocxx-v3/installation/) shows how to test it. I tried both the Compiling with the help of pkg-config and the Compiling without pkg-config or CMake and both times it successfully made an executabl, but I got the following error:
| |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 16/Jan/18 ] | |||||||||||||||||||||||||||||||
|
Probably way easier to just install the boost development packages from your systems package manager. You need to build and install boost if you get it from source - you can't just point into the downloaded files - and building it is complex and time consuming. For your other question, I have no idea why CMake is looking for pthread_create; that symbol is nowhere used inside the C++ driver sources, and we certainly didn't instruct CMake to look for it during its configure phase. I also have no idea why such a test would fail on a sanely configured system. What happens if you try to write and link your own small test program that calls pthread_create? Does it work? I'd also, if you haven't already, recommend completely removing any state in your build directory and try again. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 15/Jan/18 ] | |||||||||||||||||||||||||||||||
|
I downloaded Boost tarball from here (http://www.boost.org/users/download/) and extracted it. The headers are located in the /boost folder inside the main folder that the tarball extracts to. I set that to full path to the BOOST_INCLUDEDIR environment variable but it's still saying unable to find Boost and outputting the same error message as above^. Also it tells me to look at /mongo-cxx-driver-r3.2.0-rc0/build/CMakeFiles/CMakeError.log, and in there I see the pthread thing might be an issue:
| |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 15/Jan/18 ] | |||||||||||||||||||||||||||||||
|
I tried running CMake with the boost option. It is looking for boost headers. Do I have to install boost before I run CMake? It says:
Also, just like when I was trying to cmake with mnmlstc/core, it gives me a Looking for pthread_create - not found message. Is that an issue? Even though it gave me that message when trying to just use mnmlstc, the CMake would "work" | |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 15/Jan/18 ] | |||||||||||||||||||||||||||||||
|
I wouldn't be so sure. Build systems tend to be conservative about allowing user state affect a build. I think it is very possible that this is the root of the issue. You could confirm by setting GIT_CONFIG options in the call to ExternalProject_Add in the build system, if you are using a new enough CMake to honor that flag: https://cmake.org/cmake/help/v3.8/module/ExternalProject.html But unless you really care to dig in to why this doesn't work, go with boost? | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 15/Jan/18 ] | |||||||||||||||||||||||||||||||
|
I'm not sure what the GitHub issue is either. I've set the the proxy (http.proxy and https.proxy) in git. I would think that git checks those proxy values every time a git clone is called, whether it be manually or in a build. | |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 15/Jan/18 ] | |||||||||||||||||||||||||||||||
|
Yes, you can manually clone MNMLSTC/core and place it somewhere. In that case, you would use BSONCXX_POLY_USE_SYSTEM_MNMLSTC and if you installed it somewhere not on your toolchains default header search path you would need to pass the relevant include path argumemts to your toolchain via CMAKE_CXX_FLAGS as mentioned above. I have no idea why you can clone from the command line but not from within the build. Is it possible that your ability to communicate with GitHub is dependent on some environment variable (ssh agent, proxy config, etc.) that is not getting propagated into the build? Regarding boost - the point of the polyfills is to use what works best for you. As MNMLSTC/core and C++17 both seem to be unavailable to you, trying boost seems like the right move. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 15/Jan/18 ] | |||||||||||||||||||||||||||||||
|
I don't need mnmlstc or boost. Since a C++17 polyfill is required I will try boost and see if it lets me install. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 15/Jan/18 ] | |||||||||||||||||||||||||||||||
|
Well I can can git clone MNMLSTC/core when I manually type it in from my machine. I have the proxy set to allow for GitHub. It's just for some reason during the sudo make EP_mnmlstc_core call, when I check the EP_mnmlstc_core-download-err.log that I see: Failed to connect to github.com:443; Connection refused Failed to clone repository: 'https://github.com/mnmlstc/core' But if I do a *git clone https://github.com/mnmlstc/core*, that works. Is there a way I can just manually clone it and put it where it needs to be? | |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 12/Jan/18 ] | |||||||||||||||||||||||||||||||
|
First, I'd like to re-iterate my suggestion that you solve the issue that is preventing the driver build from doing what it normally does, which is download and install MNMLSTC/core for you. That works for almost everyone who builds the driver, unless they are in an environment that restricts access to GitHub via a firewall. If you are in such an environment, then I'd recommend building against your system boost libraries via BSONCXX_POLY_USE_BOOST, before trying to manually install MNMLSTC/core and building with BSONCXX_POLY_USE_SYSTEM_MNMLSTC. However, if you really must use MNMLSTC/core, then you will either need to install it to a location such that its header files are appropriately placed so that #include <core/string.hpp> will resolve using the compilers default header search path, or you will need to pass in additional header search paths as appropriate for the MNMLSTC/core installation to your CMake invocation by using CMAKE_CXX_FLAGS. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 12/Jan/18 ] | |||||||||||||||||||||||||||||||
|
It can't find the MNMLSTC/core files still. When I do the make && sudo make install, it tells me core/string.hpp: No such file or directory. I tried putting the internal core folder in /usr/local, mongo-cxx-3.2.0, and even the build folder but it can't find those hpp files - which I see in there. | |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 11/Jan/18 ] | |||||||||||||||||||||||||||||||
|
I don't seem to be explaining this clearly. Sorry. If you add BSONCXX_POLY_USE_SYSTEM_MNMLSTC, then the build system doesn't have any targets related to downloading, building, or installing MNMLSTC/core. So the EP_mnmlstc_core target just simply won't exist, and you don't need to build it. Just do make && sudo make install. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 11/Jan/18 ] | |||||||||||||||||||||||||||||||
|
I put added "-DBSONCXX_POLY_USE_SYSTEM_MNMLSTC=1" to the call. Now when I go try to run sudo make EP_mnmlstc_core it tells me:
Any idea why? What do you mean by the tool chain library search path? Does MNMLSTC/core have to be somewhere in /usr/local? I'm running the make in the /build folder of mongo 3.2.0. - guessing that it is only looking for a target in there. Where should I call the make? | |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 11/Jan/18 ] | |||||||||||||||||||||||||||||||
|
Did you add BSONCXX_POLY_USE_SYSTEM_MNMLSTC to your CMake invocation? Do that, and don't try to run the target which would download it. Just do a normal build. That should suppress the build system wanting to download it, and it will expect to find the MNMLSTC/core headers in the toolchain library search path: adjust as needed to reference the files you downloaded. Note that this isn't a configuration we test - we left it in as a last stopgap for people unable to download MNMLSTC/core who can't use the other polyfills. My recommendation would be to get to a root cause for why the download is failing. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 11/Jan/18 ] | |||||||||||||||||||||||||||||||
|
I'm not planning to use boost or C++17 for my project. MNMLSTC/core extracted to a folder named core and I just kept it in my ~/ directory. That's where the mongo cxx 3.2.0 folder exists as well. Still, when I run sudo make EP_mnmlstc_core I get the same error as before. It's still trying to pull MNMLSTC/core from git. Is there a way to make it look for it on the machines instead of go to GitHub? I'm surprised because this issue wasn't happening initially... | |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 09/Jan/18 ] | |||||||||||||||||||||||||||||||
|
MNMLSTC/core is header only, so just drop it wherever. The headers will need to be around when using the driver though. If you are planning on using any of boost in your project, you might do better to just use boost. Or if you can use C++17, use CMAKE_CXX_STANDARD=17 and just use std::optional and std::string_view from C++17? | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 09/Jan/18 ] | |||||||||||||||||||||||||||||||
|
I checked EP_mnmlstc_core-download-*.log, and it is an issue with being unable to access the mnmlstc/core GitHub repo. That's strange because I can clone other things from GitHub without any issues as well. Would installing manually from here (https://github.com/mnmlstc/core) work? I can't find install steps, the GitHub site links me to the documentation (http://mnmlstc.github.io/core/) which has no instal info. I am able to clone it though. | |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 09/Jan/18 ] | |||||||||||||||||||||||||||||||
|
The build tries to download MNMLSTC/core from github. If, for some reason that is blocked, you can either install MNMLSTC yourself and then build with the BSONCXX_POLY_USE_SYSTEM_MNMLSTC option, or use the boost polyfills instead, by building with BSONCXX_POLY_USE_BOOST. You will need boost 1.56 or newer available. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 09/Jan/18 ] | |||||||||||||||||||||||||||||||
|
I got 3.2.0-rc0 from here: https://github.com/mongodb/mongo-cxx-driver/releases I tried the same sudo make EP_mnmlstc_core and got the same error as above. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 09/Jan/18 ] | |||||||||||||||||||||||||||||||
|
I'm getting this error now when just trying to install the polyfill with sudo make EP_mnmlstc_core:
| |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 09/Jan/18 ] | |||||||||||||||||||||||||||||||
|
Also, this .mci.yml file (https://github.com/mongodb/mongo-cxx-driver/blob/0fde689d48a3d4a19f7635def8c48518c8c5b8ef/.mci.yml) has a bunch of differences than the one in 3.1.2 that I pulled. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 09/Jan/18 ] | |||||||||||||||||||||||||||||||
|
I just checked what type the file was that I curled (following step 3). It's an ASCII text file, it didn't grab the actual tar.gz for some reason. Also in 3.1.2 if I make the first change from the link .mci.yml, I should be good (The second change seems to already be in there)?: And what about this change?: https://github.com/mongodb/mongo-cxx-driver/commit/29a4a590c6234ab44b869df1e5de659fa15b8e52 There is no pool.cpp in /examples/mongocxx. | |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 09/Jan/18 ] | |||||||||||||||||||||||||||||||
|
3.2.0 is still pre-release but hopefully not for much longer. In the meantime, you can also just cherry-pick the fix for Also, I think you are calling tar wrong - it seems to be trying to read from stdin, not the file... | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 09/Jan/18 ] | |||||||||||||||||||||||||||||||
|
So, there is no fix for this in 3.1.2? Also, isn't 3.2.0 prelease though? Should I install it anyways? Since you can't clone 3.2.0 from github I tried downloading the tar.gz file and extracting it. It's giving me this error when I call tar -xzvf r3.2.0.tar.gz:
Is the file not a true tar.gz? I even tried renaming it so it just had the tar at the end (no gz) and it couldn't untar it. | |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 09/Jan/18 ] | |||||||||||||||||||||||||||||||
|
That issue is | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 08/Jan/18 ] | |||||||||||||||||||||||||||||||
|
Ok, great. Next, I tried to build and install the driver (Step 5 in on the site) and it went to 81% before make had some issues with "assert" not being declared "in this scope" in a file called mock.hh. Here's the error output after running 'make && sudo make install':
| |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 08/Jan/18 ] | |||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 08/Jan/18 ] | |||||||||||||||||||||||||||||||
|
I ran CMake again with the MNMLSTC option and it output this almost instantaneously:
I'm not sure if it actually worked, because the first time I ran the Cmake it took a couple of seconds. Is the MNMLSTC "default" because you don't have to name it in the call, but it's already there? Because in the guide it says to add the following option in the CMake call: -DBSONCXX_POLY_USE_MNMLSTC=1 | |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 08/Jan/18 ] | |||||||||||||||||||||||||||||||
|
No, the PKG_CONFIG_PATH thing only affects that run of CMake. Just say it again if you run CMake again. For the polyfills, yes, you need to chose. You can stick with the default of MNMLSTC, or use boost. The library will have a different ABI depending on which you chose. If you plan to use boost elsewhere in your project, it may make more sense to build the driver against boost, so that the types it uses match up nicely with whatever you are already using. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 08/Jan/18 ] | |||||||||||||||||||||||||||||||
|
So I added the package config path and it worked. Got the following output:
Should I make the PKG_CONFIG_PATH a persistent environment variable for the C++ driver to continue working? Also, I have a question about the polyfills. I won't be using anything past C++11 so do I need to select a polyfill when running the CMake? I want to know before I end up building and installing the driver. | |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 06/Jan/18 ] | |||||||||||||||||||||||||||||||
|
Can you try doing each of the following, and let me know which ones work? Try doing both, too - the results might be interesting and different:
My hope is that the first incantation will address the problem by letting CMake know where to find the pkg-config files, and that the second will address the problem by letting CMake know where to find the CMake style package files. Overall, I find this issue surprising because normally I would expect that /usr/local would be automatically searched by both of these mechanisms because you have specified it for -DCMAKE_INSTALL_PREFIX=/usr/local. Some additional information on how to configure the C++ driver to locate the C driver is in the installation notes: https://mongodb.github.io/mongo-cxx-driver/mongocxx-v3/installation/#step-4-configure-the-driver Finally, my last recommendation would be to upgrade to the latest 3.2.0-rc0 release of the driver and see if any of the above techniques fix your problems. | |||||||||||||||||||||||||||||||
| Comment by Adwait Lele [ 05/Jan/18 ] | |||||||||||||||||||||||||||||||
|
pkg-config is installed - if I run it with no arguments I get "Must specify package names on the command line" In the guide, it said that libmongoc gets installed into /usr/local by default. SO my CMake invocation was: cmake3 -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local .. Is it an issue with where libmongoc is located? Here is the output of find -iname libmongoc from /usr/local:
| |||||||||||||||||||||||||||||||
| Comment by Andrew Morrow (Inactive) [ 05/Jan/18 ] | |||||||||||||||||||||||||||||||
|
I think that pkgconfig vs pkg-config is a package naming variation, but as I understand it, the binary should always be known as pkg-config. What happens if you run pkg-config on your system where you are having the issue? I think the issue isn't with pkg-config, so much as finding libbson. Do you have the C driver installed? If so, where? How did you obtain or build it? Can you show the complete CMake invocation you used to configure the C++ driver build? |