[SERVER-10644] MongoDB not able to build from source on OS X Mavericks due to missing Boost header file Created: 29/Aug/13  Updated: 10/Dec/14  Resolved: 05/Sep/13

Status: Closed
Project: Core Server
Component/s: Build
Affects Version/s: 2.4.6
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: George Thiruvathukal Assignee: Eric Milkie
Resolution: Done Votes: 0
Labels: clang
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

$ uname -a
Darwin GT-MBP-Thb.local 13.0.0 Darwin Kernel Version 13.0.0: Sat Aug 17 20:57:00 PDT 2013; root:xnu-2422.1.63.0.1~2/RELEASE_X86_64 x86_64


Attachments: Text File patch-mavericks.diff    
Issue Links:
Related
is related to SERVER-10858 Cannot build 2.4 server source tree w... Closed
Operating System: OS X
Steps To Reproduce:

Just run scons all

Participants:

 Description   

During build, compilations fails due to the inability to find the <tr1/unordered_map> header file:

scons: Reading SConscript files ...
scons version: 2.3.0
python version: 2 7 5 'final' 0
Checking whether the C++ compiler works(cached) yes
Checking for C header file unistd.h... (cached) yes
Checking whether clock_gettime is declared... (cached) no
Checking for C++ header file execinfo.h... (cached) yes
Checking whether backtrace is declared... (cached) yes
Checking whether backtrace_symbols is declared... (cached) yes
Checking whether backtrace_symbols_fd is declared... (cached) yes
Checking for C library pcap... (cached) yes
Checking if __malloc_hook is declared volatile... (cached) no
scons: done reading SConscript files.
scons: Building targets ...
g++ -o build/darwin/normal/mongo/shell/dbshell.o -c -Wnon-virtual-dtor -Woverloaded-virtual -fPIC -fno-strict-aliasing -ggdb -pthread -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -O3 -DBOOST_ALL_NO_LIB -D_SCONS -DMONGO_EXPOSE_MACROS -DSUPPORT_UTF8 -D_FILE_OFFSET_BITS=64 -DMONGO_HAVE_HEADER_UNISTD_H -DMONGO_HAVE_EXECINFO_BACKTRACE -Ibuild/darwin/normal/third_party/libstemmer_c/include -Isrc/third_party/libstemmer_c/include -Ibuild/darwin/normal/third_party/s2 -Isrc/third_party/s2 -Ibuild/darwin/normal/third_party/boost -Isrc/third_party/boost -Ibuild/darwin/normal/third_party/pcre-8.30 -Isrc/third_party/pcre-8.30 -Ibuild/darwin/normal -Isrc -Ibuild/darwin/normal/mongo -Isrc/mongo -I/opt/local/include src/mongo/shell/dbshell.cpp
In file included from src/mongo/shell/dbshell.cpp:26:
In file included from src/mongo/base/initializer.h:21:
In file included from src/mongo/base/configuration_variable_manager.h:24:
src/mongo/platform/unordered_map.h:47:10: fatal error: 'tr1/unordered_map' file not found
#include <tr1/unordered_map>
         ^
1 error generated.
scons: *** [build/darwin/normal/mongo/shell/dbshell.o] Error 1
scons: building terminated because of errors.



 Comments   
Comment by Andrew Morrow (Inactive) [ 27/Oct/13 ]

Slava A similar fix has been committed on the master branch: https://github.com/mongodb/mongo/commit/c70a72b151fcfab26d0db5e8ceeb7c04a96ce449

Comment by Slava Bacherikov [X] [ 27/Oct/13 ]

This patch solve an issue on OS X Mavericks.

Comment by Andrew Morrow (Inactive) [ 07/Oct/13 ]

mistydemeo Also, thanks for the suggestion on _LIBCPP_VERSION. I think we may use that to tackle SERVER-11070, where libc++ doesn't currently work for us in C++03 mode.

Comment by Andrew Morrow (Inactive) [ 07/Oct/13 ]

Hi Misty -

The work to support libc++ and clang is complete, but it all took place on master after the 2.4 branch was created, and it is extremely unlikely that we would backport that work to the 2.4 branch.

You may be right that some small targeted patches against 2.4 would get it working under XCode 5 (I suspect that asking it to use libstdc++ by adding --stdlib=libstdc++ to CXXFLAGS in the Sconstruct file is the easiest way), but those patches would need to be carried along in brew/macports. The other important changes were mostly related to suppressing certain clang warnings if I remember correctly. The expectation is that MongoDB 2.6 will work with clang and libc++ out of the box.

As far as linking goes, it is true that linking with libstdc++ would make us ABI incompatible, but for the servers and tools this would not be an issue since nothing should be linking against those. The client driver is a separate issue, obviously.

Comment by Misty De Meo [ 07/Oct/13 ]

To provide a little more detail here...

clang supports two different C++ standard libraries - libstdc+, the GCC-based C+ stdlib that's traditionally been the default (also the same C++ stdlib used by Apple's GCC 4.2 and LLVM-GCC 4.2); and libc+, a newer C+ stdlib that originated in the LLVM project. The two stdlibs have been shipped with clang for a number of versions now, and are selectable using the `-stdlib=` switch, e.g. `-stdlib=libc++`.

Starting with 10.9, the default C++ stdlib is now libc+. There are a few differences between them, which is the cause of the problem here. In this case, the issue is that libc+ doesn't include the tr1-prefixed parts of the C++ stdlib, only the C++11-standardized versions of those same additions.

libc++ and libstdc++ aren't compatible, so I wouldn't recommend mixing headers from both like the proposed solution here is doing. Instead, the best solutions would be to:

  • Alter the code so mongodb compiles with libc+. It might be as simple as checking for libc+ and/or C+11 and including the non-prefixed versions of those headers, if you're confident the C11 versions are compatible. libc+ defines the _LIBCPP_VERSION macro which you can use to detect it, if an std header has been included. See, for example, this proposed patch for gearman re: the same issue - https://github.com/mxcl/homebrew/pull/22612#issuecomment-24624092
  • Add -stdlib=libstdc++ when building with clang so that you use libstdc+. However, note that libc+ and libstdc++ aren't ABI compatible, so this would limit other C++ software from linking against it.
Comment by Eric Milkie [ 05/Sep/13 ]

George, it looks like compiling with the Developer Preview version of XCode that comes with Mavericks works with MongoDB version 2.5.2. I don't think there's anything further for us to do here.

Comment by Eric Milkie [ 03/Sep/13 ]

Ah that would be it.

Comment by Andrew Morrow (Inactive) [ 03/Sep/13 ]

On master we add -Wno-unused-private-fields if supported: https://github.com/mongodb/mongo/blob/master/SConstruct#L1063.

Comment by Eric Milkie [ 03/Sep/13 ]

The unordered_map header does generate a lot of warnings, but my 2.4 build gets a lot further by using --cpppath=<etc>
The build still fails, however, on v8's store-buffer.h file:

In file included from src/third_party/v8/src/x64/assembler-x64.cc:28:
In file included from src/third_party/v8/src/v8.h:60:
In file included from src/third_party/v8/src/objects-inl.h:38:
In file included from src/third_party/v8/src/elements.h:33:
In file included from src/third_party/v8/src/heap.h:41:
src/third_party/v8/src/store-buffer.h:229:9: error: private field 'heap_' is not used [-Werror,-Wunused-private-field]
  Heap* heap_;
        ^

I haven't determined why we don't have this problem on master branch.

Comment by Eric Milkie [ 03/Sep/13 ]

I discovered that you can easily see which directories are being searched for the include path by using "-v":

$ g++ -v test.cpp
Apple LLVM version 5.0 (clang-500.1.74) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin13.0.0
Thread model: posix
 "/Library/Developer/CommandLineTools/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.9.0 -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -main-file-name test.cpp -mrelocation-model pic -pic-level 2 -mdisable-fp-elim -masm-verbose -munwind-tables -target-cpu core2 -target-linker-version 143 -v -resource-dir /Library/Developer/CommandLineTools/usr/bin/../lib/clang/5.0 -stdlib=libc++ -fdeprecated-macro -fdebug-compilation-dir /Users/yellow -ferror-limit 19 -fmessage-length 150 -stack-protector 1 -mstackrealign -fblocks -fobjc-runtime=macosx-10.9.0 -fobjc-dispatch-method=mixed -fobjc-default-synthesize-properties -fencode-extended-block-signature -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o /var/folders/v6/jncfvj9x2811p_638ht1jjfh0000gn/T/test-N5f12a.o -x c++ test.cpp
clang -cc1 version 5.0 based upon LLVM 3.3svn default target x86_64-apple-darwin13.0.0
ignoring nonexistent directory "/usr/include/c++/v1"
#include "..." search starts here:
#include <...> search starts here:
 /Library/Developer/CommandLineTools/usr/bin/../lib/c++/v1
 /usr/local/include
 /Library/Developer/CommandLineTools/usr/bin/../lib/clang/5.0/include
 /Library/Developer/CommandLineTools/usr/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
test.cpp:1:10: fatal error: 'tr1/unordered_map' file not found
#include <tr1/unordered_map>
         ^
1 error generated.

So despite being configured with --with-gxx-include-dir=/usr/include/c++/4.2.1, you apparently must still manually tell the compiler to amend its include path.
We'll figure out how to fix this properly in master, but in the 2.4 series you can try building with the --cpppath parameter to scons.

To diagnose why scons' config stage isn't finding a boost header, look in .scons/<platform>/config.log to see why the test file didn't compile.

Comment by George Thiruvathukal [ 30/Aug/13 ]

Oh, yeah, this is very interesting. The executables are 'technically' different - but not very different:

$ ls -l $(which g++ clang)
-rwxr-xr-x  1 root  wheel  14224 Aug 21 21:17 /usr/bin/clang*
-rwxr-xr-x  1 root  wheel  14224 Aug 21 21:17 /usr/bin/g++*
$ md5 $(which g++ clang)
MD5 (/usr/bin/g++) = df7d266e76ec30c091ef1d462e72ab34
MD5 (/usr/bin/clang) = 5b5c8153dcc6be4d299aad764bb8b87d
$ file $(which g++ clang)
/usr/bin/g++:   Mach-O 64-bit executable x86_64
/usr/bin/clang: Mach-O 64-bit executable x86_64
$ otool -L $(which g++ clang)
/usr/bin/g++:
	/usr/lib/libxcselect.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)
/usr/bin/clang:
	/usr/lib/libxcselect.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)

And clang fails as well to compile the test code:

$ clang test.cpp 
test.cpp:1:10: fatal error: 'tr1/unordered_map' file not found
#include <tr1/unordered_map>
         ^
1 error generated.

Comment by Andrew Morrow (Inactive) [ 29/Aug/13 ]

Very interesting.

  • It looks like Apple really have finally done it and are no longer shipping GCC at all in Mavericks. Both 'g++' and 'clang' on your machine are really clang.
  • The 'g++' clang appears to have been differently configured to search /usr/include/c++/4.2.1. Is there any sort of symlink relationship between the g++ and clang binaries? Or are they really distinct?
  • The --with-gxx-include-dir configure hackery doesn't seem to actually work per your needing to explicitly add that path to make the TR1 headers available.
  • What happens if you try to compile the trivial example program with 'clang' instead of g++?

I'll need to sit down with a Mavericks + XCode 5 setup to explore further, but my quick analysis is that there is some sort of Apple side bug related to the 'fake' g++ not correctly searching for tr1 headers in the configured 'gxx-include-dir'. That example program you tested out really needs to work. That may need to be reported upstream.

However, even if you were able to get past that issue, I think the bigger problem you face is that mongodb 2.4 offers incomplete support for clang, so I'd expect that you will run into other problems if you get past this. However, you should be able to build mongodb 2.4 on an older platform and run it on Mavericks, assuming the system dependencies are met.

Please note that we are working on improved clang support in the 2.5 development release of mongodb. However, I'm not aware of Mavericks support currently being an explicit release goal for 2.6.

Comment by George Thiruvathukal [ 29/Aug/13 ]

Sure. Here's clang:

$ clang --version
Apple LLVM version 5.0 (clang-500.1.74) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin13.0.0
Thread model: posix

Comment by Andrew Morrow (Inactive) [ 29/Aug/13 ]

I think we are getting somewhere now. Your 'g++' isn't GCC: it is clang. Can you verify by seeing what the output of 'clang --version' is? I bet it is identical to what you get for 'g++ --version'.

Comment by George Thiruvathukal [ 29/Aug/13 ]

Yes, g++ does compile the test code when specifying the include path:

$ cat test.cpp 
#include <tr1/unordered_map>
int main() { return 0; }
$ g++ test.cpp 
test.cpp:1:10: fatal error: 'tr1/unordered_map' file not found
#include <tr1/unordered_map>
         ^
1 error generated.
$ g++ -I /usr/include/c++/4.2.1 test.cpp 
$ echo $?
0
$ ls -l a.out 
-rwxr-xr-x  1 gthiruva  staff  4352 Aug 29 12:56 a.out*
$ ./a.out

Here's the version info:

$ g++ --version
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 5.0 (clang-500.1.74) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin13.0.0
Thread model: posix

And one additional fun fact:

$ scons --use-system-boost all
scons: Reading SConscript files ...
scons version: 2.3.0
python version: 2 7 5 'final' 0
Checking whether the C++ compiler worksyes
Checking for C++ header file boost/filesystem/operations.hpp... no
can't find boost headers

Comment by Andrew Morrow (Inactive) [ 29/Aug/13 ]

Can your g++ compile a simple test file containing:

#include <tr1/unordered_map>
int main() { return 0; }

Can you also provide the output of 'g++ --version'?

Comment by George Thiruvathukal [ 29/Aug/13 ]

Sure can. Here you go, Andrew:

$ mdfind -name unordered_map
/Library/Developer/CommandLineTools/usr/lib/c++/v1/unordered_map
/usr/include/c++/4.2.1/tr1/unordered_map

$ pkgutil --file-info /Library/Developer/CommandLineTools/usr/lib/c++/v1/unordered_map
volume: /
path: /Library/Developer/CommandLineTools/usr/lib/c++/v1/unordered_map
 
pkgid: com.apple.pkg.CLTools_Executables
pkg-version: 5.0.0.0.1.1376907989
install-time: 1377548944
uid: 0
gid: 80
mode: 644

$ pkgutil --file-info /usr/include/c++/4.2.1/tr1/unordered_map
volume: /
path: /usr/include/c++/4.2.1/tr1/unordered_map
 
pkgid: com.apple.pkg.MacOSX10_9_SDK
pkg-version: 5.0.0.0.1.1376907989
install-time: 1377548944
uid: 0
gid: 0
mode: 644

Comment by Andrew Morrow (Inactive) [ 29/Aug/13 ]

On my Mountain Lion machine, I find GCC's tr1/unordered_map to exist as /usr/include/c++/4.2.1/tr1/unordered_map, provided by package com.apple.pkg.DevSDK:

$ mdfind -name unordered_map | grep /usr/include
/usr/include/c++/4.2.1/tr1/unordered_map
 
$ pkgutil --file-info /usr/include/c++/4.2.1/tr1/unordered_map
volume: /
path: /usr/include/c++/4.2.1/tr1/unordered_map
 
pkgid: com.apple.pkg.DevSDK
pkg-version: 10.8.0.0.1.1306847324
install-time: 1367464634
uid: 0
gid: 0
mode: 644

Unfortunately I don't have a Mavericks machine handy to test with right now. Can you check what unordered_map files are available on your machine and post the list back here?

Thanks,
Andrew

Generated at Thu Feb 08 03:23:42 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.