[SERVER-19661] Build fails: error: expected expression Created: 30/Jul/15  Updated: 13/Oct/15  Resolved: 08/Sep/15

Status: Closed
Project: Core Server
Component/s: Build
Affects Version/s: None
Fix Version/s: 3.0.7, 3.1.8

Type: Bug Priority: Major - P3
Reporter: Ryan Schmidt Assignee: Jonathan Reams
Resolution: Done Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File build.log     File mongodb-3.0.4-main.log.bz2     File mongodb-3.0.5-main.log.bz2    
Issue Links:
Duplicate
duplicates SERVER-19755 scons should require c++11 on 3.0 Closed
Related
related to SERVER-19755 scons should require c++11 on 3.0 Closed
related to SERVER-20712 Build failure with --use-cpu-profiler... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Completed:
Steps To Reproduce:

Build mongodb 3.0.5 on OS X

Sprint: Build 7 08/10/15, Build 8 08/31/15, Build 9 (09/18/15)
Participants:

 Description   

Hello, I'm the maintainer of mongodb in MacPorts. I'm unable to build mongodb 3.0.5. This is the error I see:

Generating placeholder library build/darwin/64/cc__usr_bin_clang/cxx__usr_bin_clang++/osx-version-min_10.10/ssl/use-system-boost/use-system-pcre/use-system-snappy/mongo/libserver_options_core.a
Skipping ranlib for build/darwin/64/cc__usr_bin_clang/cxx__usr_bin_clang++/osx-version-min_10.10/ssl/use-system-boost/use-system-pcre/use-system-snappy/mongo/libserver_options_core.a
/usr/bin/clang++ -o build/darwin/64/cc__usr_bin_clang/cxx__usr_bin_clang++/osx-version-min_10.10/ssl/use-system-boost/use-system-pcre/use-system-snappy/mongo/shell/bench.o -c -Wnon-virtual-dtor -Woverloaded-virtual -fPIC -fno-strict-aliasing -ggdb -pthread -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -pipe -O3 -m64 -Wno-unused-function -Wno-unused-private-field -Wno-deprecated-declarations -Wno-tautological-constant-out-of-range-compare -Wno-unused-const-variable -Wno-missing-braces -mmacosx-version-min=10.10 -D_SCONS -DMONGO_EXPOSE_MACROS -DPCRE_STATIC -DSUPPORT_UTF8 -DMONGO_OPTIMIZED_BUILD -DMONGO_BYTE_ORDER=1234 -D_FILE_OFFSET_BITS=64 -DMONGO_SSL -DMONGO_HAVE___THREAD -DMONGO_HAVE_GCC_ATOMIC_BUILTINS -DBOOST_SYSTEM_NO_DEPRECATED -DBOOST_THREAD_VERSION=2 -DMONGO_HAVE_HEADER_UNISTD_H -DMONGO_HAVE_EXECINFO_BACKTRACE -DMONGO_HAVE_FIPS_MODE_SET -I/opt/local/include -Isrc/third_party/s2 -Ibuild/darwin/64/cc__usr_bin_clang/cxx__usr_bin_clang++/osx-version-min_10.10/ssl/use-system-boost/use-system-pcre/use-system-snappy -Isrc src/mongo/shell/bench.cpp
src/mongo/shell/bench.cpp:379:17: warning: 'auto' type specifier is a C++11 extension [-Wc++11-extensions]
                auto& stats = shouldCollectStats() ? _stats : _statsBlackHole;
                ^
src/mongo/shell/bench.cpp:914:10: warning: 'auto' type specifier is a C++11 extension [-Wc++11-extensions]
         auto appendPerSec = [&buf, runner](StringData name, double total) {
         ^
src/mongo/shell/bench.cpp:914:30: error: expected expression
         auto appendPerSec = [&buf, runner](StringData name, double total) {
                             ^
2 warnings and 1 error generated.
scons: *** [build/darwin/64/cc__usr_bin_clang/cxx__usr_bin_clang++/osx-version-min_10.10/ssl/use-system-boost/use-system-pcre/use-system-snappy/mongo/shell/bench.o] Error 1
scons: building terminated because of errors.
Command failed:  cd "/opt/local/var/macports/build/_Users_rschmidt_macports_dports_databases_mongodb/mongodb/work/mongodb-src-r3.0.5-x86_64" && /opt/local/bin/scons all mongobridge mongosniff --cc=/usr/bin/clang --cxx=/usr/bin/clang++ --cpppath=/opt/local/include --disable-warnings-as-errors --libpath=/opt/local/lib --osx-version-min=10.10 --ssl --system-boost-lib-search-suffixes=-mt --use-system-boost --use-system-pcre --use-system-snappy --extralib=pcre,pcrecpp,snappy --64 

3.0.4 builds fine on the same system, which has:

$ xcodebuild -version
Xcode 6.4
Build version 6E35b

$ clang -v
Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
Target: x86_64-apple-darwin14.5.0
Thread model: posix

$ sw_vers
ProductName: Mac OS X
ProductVersion: 10.10.5
BuildVersion: 14F6a

I've attached logs of an unsuccessful build of 3.0.5 and a successful build of 3.0.4.



 Comments   
Comment by Jonathan Reams [ 08/Sep/15 ]

Great, it will be in the next version of mongodb. We can re-open this if you have any more trouble.

Comment by Ryan Schmidt [ 08/Sep/15 ]

Thanks. I had also come to the conclusion that the flags for the third party libraries needed to come before, not after, the extra cpp and lib paths, but was having trouble understanding your build system to make that happen. Your patch works for me.

Comment by Jonathan Reams [ 01/Sep/15 ]

Ryan, sorry for taking so long to get back to you. I just pushed a change that will be in the next 3.0 release that I think should fix your problem. Before now we've appended our third-party libraries to the include search-path, which meant that anything specified with --cpppath took precedence over our bundled libraries. Now you should be able to use --cpppath=/opt/local/include --libpath=/opt/local/lib and not pull in the system boost by accident. If you want to test it you can apply this commit (https://github.com/mongodb/mongo/commit/8b09ad789b4f156141441b4823aae48fb0eb8de7). Let me know if you have any questions or trouble.

Comment by Githook User [ 01/Sep/15 ]

Author:

{u'username': u'jbreams', u'name': u'Jonathan Reams', u'email': u'jbreams@mongodb.com'}

Message: SERVER-19661 Give vendored include paths higher priority
Branch: v3.0
https://github.com/mongodb/mongo/commit/8b09ad789b4f156141441b4823aae48fb0eb8de7

Comment by Githook User [ 01/Sep/15 ]

Author:

{u'username': u'jbreams', u'name': u'Jonathan Reams', u'email': u'jbreams@mongodb.com'}

Message: SERVER-19661 Give vendored include paths higher priority
Branch: master
https://github.com/mongodb/mongo/commit/f725d04705a2019ab1949ced290158397d9365b0

Comment by Ryan Schmidt [ 19/Aug/15 ]

I initially thought that boost was the only C++ using library mongodb used, but I see now that pcre and snappy also use C++, so we would have to switch to using the bundled versions of those as well. I'm happy to do that.

Previously, we have been using the scons flags --cpppath=/opt/local/include, --libpath=/opt/local/lib, --ssl, --system-boost-lib-search-suffixes=-mt, --use-system-boost, --use-system-pcre, --use-system-snappy, --extralib=pcre,pcrecpp,snappy, which was working fine for using the MacPorts versions of boost, pcre, snappy, openssl and libpcap.

If I remove the above flags except --ssl, then the bundled versions of boost, pcre and snappy are used, which is desired, but also the OS X versions of openssl and libpcap get used instead of the newer MacPorts versions, which is not desired.

If I leave --cpppath=/opt/local/include and --libpath=/opt/local/lib, then the build fails because some MacPorts boost headers get used instead of the bundled ones.

What is the correct way to use the bundled boost, pcre and snappy libraries, while still using the openssl and libpcap libraries in MacPorts?

Comment by Andrew Morrow (Inactive) [ 12/Aug/15 ]

ryandesign -

Yes, that is correct. The C++ driver was moved to a separate project (the "legacy" C++ driver), and that project intentionally does not bundle any libraries. It is intended that the legacy C++ driver be built against the system libraries, and it does not and will not require C++11. So, mongodb and the C++ driver can definitely use different versions of boost. Sometime in the next several months we will be releasing a new C++ driver, which will require C++11, but this driver will not have any third party C++ library dependencies.

Please note that there are some other C++ libraries that the server bundles. For those that offer an option to use the system version (look in the top-of-tree SConstruct file for --use-system-x flags), it is critical that either you use the bundled library, or that if you use the system version of a library that the library is built with the same C++ runtime and language conformance level as the server sources.

Comment by Ryan Schmidt [ 12/Aug/15 ]

The reason we are building with MacPorts boost is that we were previously asked to do so by a user who was trying to use the libmongoclient.a library:

https://trac.macports.org/ticket/38334

But it seems that mongodb no longer installs libmongoclient.a nor any library. Is that correct? And boost appears to be the only C++ library mongodb uses. If these are true, then it should be possible to switch mongodb back to using its internal copy of boost and set it to always use libc++ even on older OS X systems.

It looks like libmongoclient.a is now in the separate mongo-cxx-driver project. Is it a problem if mongodb uses its internal copy of boost and mongo-cxx-driver uses a different (probably newer) MacPorts version of boost?

It looks like mongo-cxx-driver doesn't use C++11 yet, because it still builds ok on older OS X versions.

Comment by Jonathan Reams [ 10/Aug/15 ]

Unfortunately it's not going to be possible to build mongodb on OS X without libc++ going forward. We've started allowing C++11 code into 3.0 recently, but 3.1 has allowed C++11 from the start and it is used throughout the codebase. Building with any system copies of our vendored C++ libraries that weren't built with C++11 is going to be unsafe at best going forward. Is there a specific reason you're building with the MacPorts copy of boost?

Comment by Ryan Schmidt [ 06/Aug/15 ]

Perhaps that would work if we were using the copy of boost bundled with mongodb, but we're not; we were previously asked by a MacPorts user to use the MacPorts copy of boost, which means that it too must be built with libc++, which would then impact everything else in MacPorts using boost, and so on. The decision about whether to use libc++ or libstdc++ must be made for the MacPorts installation as a whole and cannot be made for individual programs.

Comment by Jonathan Reams [ 06/Aug/15 ]

On OSX 10.7 and 10.8 SCons should be run with the --libc++ flag to tell clang to use the correct C++11 library. SCons will also enforce a minimum XCode version of 5.1.1. Our internal build farm is running 10.8, so this should work fine. Looking at your link, 10.7 should work the same way. We also set the --osx-version-min=10.7 flag to make sure we maintain compatibility with 10.7 in our OSX builds.

Comment by Ryan Schmidt [ 06/Aug/15 ]

Thanks, adding --c++11 does work on OS X 10.9 and later, but please understand that this new C++11 requirement makes compilation of mongodb 3.0.5 on OS X 10.7 an 10.8 more difficult than compiling 3.0.4 and earlier, since the default C++ library on 10.8 and earlier does not support C++11. See:

https://trac.macports.org/wiki/LibcxxOnOlderSystems

Comment by Jonathan Reams [ 04/Aug/15 ]

Yes, I'll open a ticket to make sure scons requires c++11 from now on (SERVER-19755).

Comment by Ultrabug [ 04/Aug/15 ]

Thanks Jonathan, this indeed fixes this issue (tho we should expect a scons build config update). Packaged for Gentoo Linux !

Comment by Jonathan Reams [ 04/Aug/15 ]

Hi Ryan, can you try adding --c++11 to your scons command line? We've recently allowed C++11 into the 3.0 branch, but it looks like scons still needs to be updated to require C++11 to build - right now it's set to auto.

Comment by Ultrabug [ 03/Aug/15 ]

This is a build log failing also with system boot-1.56 so I guess this is not related to 1.57.

Comment by Ultrabug [ 30/Jul/15 ]

Same here for Gentoo Linux.

Looks like I can build fine only if I use the bundled boost version (so by removing the --use-system-boost).

Boost : dev-libs/boost-1.57.0 USE="nls python threads -context -debug -doc -icu -mpi -static-libs -tools"

Generated at Thu Feb 08 03:51:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.