[SERVER-77007] MongoDB fails to compile with Clang 15 with C++20 Created: 10/May/23  Updated: 29/Oct/23  Resolved: 27/Jul/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.1.0-rc0

Type: Bug Priority: Major - P3
Reporter: Mark Benvenuto Assignee: Colin Stolley
Resolution: Fixed Votes: 0
Labels: query-director-triage
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-77161 server fails to build with macosx 13.... Closed
Assigned Teams:
Query Execution
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:

 Description   

MongoDB fails to compile with newer versions of clang while in C++20 mode. The issue is that there is a circular dependency in the mongo::Value type which is incompatible with the newly constexpr std::vector constructor.

The mongo::Value type depends on mongo::ValueStorage which depends on mongo::RCVector which depends on a forward declaration of mongo::Value.

Environments:
Linux, Clang 15 and Clang 16
MacOS, Clang 14.0.3, SDK 13.3

 clang --version
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: arm64-apple-darwin22.1.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Command Line:

-o build/local/mongo/bson/bson_validate.o -c -Werror=unused-result -Woverloaded-virtual -Wpessimizing-move -Wno-undefined-var-template -Wno-instantiation-after-specialization -fsized-deallocation -Wno-defaulted-function-deleted -Wunused-exception-parameter -Wno-deprecated -std=c++20 -Wno-error=unknown-warning-option -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk -mmacosx-version-min=11.0 --target=darwin20.0.0 -arch arm64 -Werror -include mongo/platform/basic.h -ffp-contract=off -fasynchronous-unwind-tables -g2 -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -fno-omit-frame-pointer -fno-strict-aliasing -O0 -Wno-unused-local-typedefs -Wno-unused-function -Wno-unused-private-field -Wno-deprecated-declarations -Wno-tautological-constant-out-of-range-compare -Wno-tautological-constant-compare -Wno-tautological-unsigned-zero-compare -Wno-tautological-unsigned-enum-zero-compare -Wno-unused-const-variable -Wno-unused-but-set-variable -Wno-missing-braces -Wno-inconsistent-missing-override -Wno-potentially-evaluated-expression -Wno-psabi -Wno-unused-lambda-capture -Wunguarded-availability -fstack-protector-strong -fno-limit-debug-info -fdiagnostics-color -MMD -MF build/local/mongo/bson/bson_validate.o.d -Wimplicit-fallthrough -DSAFEINT_USE_INTRINSICS=0 -DPCRE2_STATIC -DABSL_FORCE_ALIGNED_ACCESS -DBOOST_ENABLE_ASSERT_DEBUG_HANDLER -DBOOST_FILESYSTEM_NO_CXX20_ATOMIC_REF -DBOOST_LOG_NO_SHORTHAND_NAMES -DBOOST_LOG_USE_NATIVE_SYSLOG -DBOOST_LOG_WITHOUT_THREAD_ATTR -DBOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS -DBOOST_SYSTEM_NO_DEPRECATED -DBOOST_THREAD_USES_DATETIME -DBOOST_THREAD_VERSION=5 -Isrc/third_party/variant-1.4.0/include -Isrc/third_party/libstemmer_c/include -Isrc/third_party/SafeInt -Isrc/third_party/murmurhash3 -Isrc/third_party/immer/dist -Isrc/third_party/fmt/dist/include -Isrc/third_party/boost -Isrc/third_party/abseil-cpp/dist -Ibuild/local -Isrc src/mongo/bson/bson_validate.cp

Output:

➜  mongo git:(fips_mac) ✗ ninja -j 1  -d keeprsp
[1/4883] Compiled build/local/mongo/bson/bson_validate.o
FAILED: build/local/mongo/bson/bson_validate.o
rm -f build/local/mongo/bson/bson_validate.o; export PATH='/usr/local/bin:/opt/bin:/bin:/usr/bin';export PATHOSX='/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/usr/local/go/bin:/opt/X11/bin:~/.dotnet/tools:/Library/Apple/usr/bin:/Applications/Wireshark.app/Contents/MacOS';/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ @build/local/mongo/bson/bson_validate.o.rsp
In file included from src/mongo/bson/bson_validate.cpp:30:
In file included from src/mongo/bson/bson_validate.h:34:
In file included from src/mongo/base/status.h:33:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/string:551:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/string_view:223:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/functional:515:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/__functional/boyer_moore_searcher.h:26:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/vector:540:52: error: arithmetic on a pointer to an incomplete type 'mongo::Value'
        {return static_cast<size_type>(__end_cap() - this->__begin_);}
                                       ~~~~~~~~~~~ ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/vector:760:56: note: in instantiation of member function 'std::vector<mongo::Value>::capacity' requested here
      __annotate_contiguous_container(data(), data() + capacity(),
                                                       ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/vector:431:7: note: in instantiation of member function 'std::vector<mongo::Value>::__annotate_delete' requested here
      __annotate_delete();
      ^
src/mongo/db/exec/document_value/value_internal.h:54:5: note: in instantiation of member function 'std::vector<mongo::Value>::~vector' requested here
    RCVector() {}
    ^
src/mongo/db/exec/document_value/value_internal.h:48:7: note: forward declaration of 'mongo::Value'
class Value;
      ^
In file included from src/mongo/bson/bson_validate.cpp:30:
In file included from src/mongo/bson/bson_validate.h:34:
In file included from src/mongo/base/status.h:33:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/string:551:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/string_view:223:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/functional:515:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/__functional/boyer_moore_searcher.h:26:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/vector:760:54: error: arithmetic on a pointer to an incomplete type 'const std::vector<mongo::Value>::value_type' (aka 'const mongo::Value')
      __annotate_contiguous_container(data(), data() + capacity(),
                                              ~~~~~~ ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/vector:431:7: note: in instantiation of member function 'std::vector<mongo::Value>::__annotate_delete' requested here
      __annotate_delete();
      ^
src/mongo/db/exec/document_value/value_internal.h:54:5: note: in instantiation of member function 'std::vector<mongo::Value>::~vector' requested here
    RCVector() {}
    ^
src/mongo/db/exec/document_value/value_internal.h:48:7: note: forward declaration of 'mongo::Value'
class Value;
      ^
In file included from src/mongo/bson/bson_validate.cpp:30:
In file included from src/mongo/bson/bson_validate.h:34:
In file included from src/mongo/base/status.h:33:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/string:551:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/string_view:223:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/functional:515:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/__functional/boyer_moore_searcher.h:26:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/vector:833:64: error: arithmetic on a pointer to an incomplete type 'mongo::Value'
        __alloc_traits::destroy(__alloc(), _VSTD::__to_address(--__soon_to_be_end));
                                                               ^ ~~~~~~~~~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/vector:827:29: note: in instantiation of member function 'std::vector<mongo::Value>::__base_destruct_at_end' requested here
  void __clear() _NOEXCEPT {__base_destruct_at_end(this->__begin_);}
                            ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/vector:436:9: note: in instantiation of member function 'std::vector<mongo::Value>::__clear' requested here
        __clear();
        ^
src/mongo/db/exec/document_value/value_internal.h:54:5: note: in instantiation of member function 'std::vector<mongo::Value>::~vector' requested here
    RCVector() {}
    ^
src/mongo/db/exec/document_value/value_internal.h:48:7: note: forward declaration of 'mongo::Value'
class Value;
      ^
In file included from src/mongo/bson/bson_validate.cpp:30:
In file included from src/mongo/bson/bson_validate.h:34:
In file included from src/mongo/base/status.h:33:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/string:549:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/memory:848:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/__memory/allocator.h:128:58: error: invalid application of 'sizeof' to an incomplete type 'mongo::Value'
            _VSTD::__libcpp_deallocate((void*)__p, __n * sizeof(_Tp), _LIBCPP_ALIGNOF(_Tp));
                                                         ^~~~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/__memory/allocator_traits.h:282:13: note: in instantiation of member function 'std::allocator<mongo::Value>::deallocate' requested here
        __a.deallocate(__p, __n);
            ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/c++/v1/vector:437:25: note: in instantiation of member function 'std::allocator_traits<std::allocator<mongo::Value>>::deallocate' requested here
        __alloc_traits::deallocate(__alloc(), this->__begin_, capacity());
                        ^
src/mongo/db/exec/document_value/value_internal.h:54:5: note: in instantiation of member function 'std::vector<mongo::Value>::~vector' requested here
    RCVector() {}
    ^
src/mongo/db/exec/document_value/value_internal.h:48:7: note: forward declaration of 'mongo::Value'
class Value;
      ^
4 errors generated.
ninja: build stopped: subcommand failed.



 Comments   
Comment by Githook User [ 27/Jul/23 ]

Author:

{'name': 'Colin Stolley', 'email': 'cstolley@gmail.com', 'username': 'ccstolley'}

Message: SERVER-77007: MongoDB fails to compile with Clang 15 with C++20
Branch: master
https://github.com/mongodb/mongo/commit/688f62c2904a8630acf44298bb040fce787386cf

Comment by Colin Stolley [ 27/Jul/23 ]

Created SERVER-79426 as a follow-up for problems masked by this problem.

Comment by Mark Benvenuto [ 11/Jul/23 ]

Yes, it still repros with fd6296cfb951f0593d94ae512c795f02dccf729f (a commit from today on master). It will likely block the next toolchain upgrade in the future.

Comment by Kyle Suarez [ 11/Jul/23 ]

mark.benvenuto@mongodb.com is this still an active issue that needs to be fixed?

Comment by Ivan Fefer [ 13/Jun/23 ]

As I understand, when we instantiate std::vector<Value>, we need to compile all the constructors of this class. Even possibly unused ones. So newly added constexpr constructor is still compiled even if it is not used (possibly removed later by the optimizer).

But also, I am not sure I understand how std::vector of forward-declared type works now, as we also need to know the size of the object to allocate memory correctly. 

Comment by David Storch [ 16/May/23 ]

kyle.suarez@mongodb.com bernard.gorman@mongodb.com this looks like something that someone on the team should take a look at. Putting it into query-director-triage.

Comment by Daniel Moody [ 16/May/23 ]

Talking with mark.benvenuto@mongodb.com, I don't understand where we are making constexpr std::vector<Value> declaration. Yes, the latest c++ implementations have the ability to create constexpr std::vectors but that doesn't mean every std::vector must be constexpr. The error message is not clear of the origins of the issue, but some constexpr declaration somewhere may in turn lead to the constexpr std::vector<Value> that the error message is complaining about.

However I still have some reservations about this, and feel there may be a bug in the libc++ implementation where its mistakenly trying to make a constexpr std::vector<Value> when our code did not declare such. Even if thats not the case, the fact that the compiler did not point to the real origins of such a declaration in its error message which caused this issue could be considered a bug as well.

Generated at Thu Feb 08 06:34:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.