[SERVER-77694] cannot compile `immer` header with `--opt=off` Created: 01/Jun/23  Updated: 18/Jan/24  Resolved: 07/Jun/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.1.0-rc0, 7.0.0-rc3, 5.0.25, 6.0.14

Type: Bug Priority: Major - P3
Reporter: Billy Donahue Assignee: Henrik Edin
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File cmd.sh     Text File cmd.txt    
Issue Links:
Backports
Depends
is depended on by SERVER-78430 create a clean fix for immer exceptio... Open
Assigned Teams:
Storage Execution
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v7.0, v6.0, v5.0
Sprint: Execution Team 2023-06-12
Participants:

 Description   

Immer incorrectly assumes GCC does not support exceptions, because it's checking a clang-specific feature macro.

Furthermore, with optimization turned off, its IMMER_RETHROW empty definition yields a function missing a return statement, a fatal build warning for us.

Scons script and output attached.

Essentially this scons invocation:

$ python3-venv/bin/python3 \
   buildscripts/scons.py \
   VARIANT_DIR=enterprise_dynamic_gcc_gdb_noopt_asan \
   LLVM_SYMBOLIZER=/opt/mongodbtoolchain/v4/bin/llvm-symbolizer \
   VERBOSE=1 \
   LIBS=rt \
   --variables-files=etc/scons/mongodbtoolchain_stable_gcc.vars \
   --sanitize=address \
   --allocator=system \
   --link-model=dynamic \
   --opt=off \
   build/enterprise_dynamic_gcc_gdb_noopt_asan/mongo/db/catalog/historical_catalogid_tracker.dyn.o

Yields a series of fatal warnings:

/opt/mongodbtoolchain/v4/bin/g++ -o build/enterprise_dynamic_gcc_gdb_noopt_asan/mongo/db/catalog/historical_catalogid_tracker.dyn.o -c -Woverloaded-virtual -Wpessimizing-move -Wno-maybe-uninitialized -fsized-deallocation -std=c++20 -Werror -include mongo/platform/basic.h -ffp-contract=off -fasynchronous-unwind-tables -g2 -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -gdwarf-5 -fno-omit-frame-pointer -fno-strict-aliasing -O0 -march=sandybridge -mtune=generic -mprefer-vector-width=128 -Wno-unused-local-typedefs -Wno-unused-function -Wno-deprecated-declarations -Wno-unused-const-variable -Wno-unused-but-set-variable -Wno-missing-braces -Wno-psabi -fstack-protector-strong -fsanitize=address -fno-omit-frame-pointer -gdwarf32 -Wa,--nocompress-debug-sections -fno-builtin-memcmp -Wimplicit-fallthrough=5 -fPIC -DSAFEINT_USE_INTRINSICS=0 -DPCRE2_STATIC -DMONGO_USE_VISIBILITY -DNDEBUG -D_XOPEN_SOURCE=700 -D_GNU_SOURCE -DADDRESS_SANITIZER -DABSL_FORCE_ALIGNED_ACCESS -DBOOST_ENABLE_ASSERT_DEBUG_HANDLER -DBOOST_FILESYSTEM_NO_CXX20_ATOMIC_REF -DBOOST_LOG_NO_SHORTHAND_NAMES -DBOOST_LOG_USE_NATIVE_SYSLOG -DBOOST_LOG_WITHOUT_THREAD_ATTR -DBOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS -DBOOST_SYSTEM_NO_DEPRECATED -DBOOST_THREAD_USES_DATETIME -DBOOST_THREAD_VERSION=5 -DBOOST_LOG_DYN_LINK -Isrc/third_party/s2 -Isrc/third_party/variant-1.4.0/include -Isrc/third_party/libstemmer_c/include -Isrc/third_party/SafeInt -Isrc/third_party/murmurhash3 -Isrc/third_party/immer/dist -Isrc/third_party/fmt/dist/include -Isrc/third_party/boost -Isrc/third_party/abseil-cpp/dist -Ibuild/enterprise_dynamic_gcc_gdb_noopt_asan -Isrc src/mongo/db/catalog/historical_catalogid_tracker.cpp
In file included from src/third_party/immer/dist/immer/map.hpp:12,
                 from src/mongo/util/immutable/unordered_map.h:32,
                 from src/mongo/db/catalog/historical_catalogid_tracker.h:34,
                 from src/mongo/db/catalog/historical_catalogid_tracker.cpp:30:
src/third_party/immer/dist/immer/detail/hamts/champ.hpp: In member function 'immer::detail::hamts::champ<T, Hash, Equal, MemoryPolicy, B>::add_result immer::detail::hamts::champ<T, Hash, Equal, MemoryPolicy, B>::do_add(immer::detail::hamts::champ<T, Hash, Equal, MemoryPolicy, B>::node_t*, T, immer::detail::hamts::hash_t, immer::detail::hamts::shift_t) const [with T = mongo::NamespaceString; Hash = absl::lts_20211102::hash_internal::Hash<mongo::NamespaceString>; Equal = std::equal_to<mongo::NamespaceString>; MemoryPolicy = immer::memory_policy<immer::heap_policy<immer::cpp_heap>, immer::refcount_policy, void, immer::no_transience_policy, true, true>; unsigned int B = 5]':
src/third_party/immer/dist/immer/detail/hamts/champ.hpp:620:5: error: control reaches end of non-void function [-Werror=return-type]
  620 |     }
      |     ^



 Comments   
Comment by Githook User [ 18/Jan/24 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-77694 Import immer lib from fork containing fix to always enable exceptions

(cherry picked from commit 016bb8a5ecc4b91e0a6fdcc58d0ff66c238fa970)

GitOrigin-RevId: b1eaa84c50588356f31c6da736f8dee2f2d5e26a
Branch: v5.0
https://github.com/mongodb/mongo/commit/1c426ee44ad16caa8c5ccd79edebe41a213a9238

Comment by Githook User [ 17/Jan/24 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-77694 Import immer lib from fork containing fix to always enable exceptions

(cherry picked from commit 016bb8a5ecc4b91e0a6fdcc58d0ff66c238fa970)

GitOrigin-RevId: 8651ee570081d7fa33a44a53207494a4d390fcdf
Branch: v6.0
https://github.com/mongodb/mongo/commit/fb8ba242265f6443653d170aadd84c7e86375cc4

Comment by Billy Donahue [ 26/Jun/23 ]

Our upstream issue report received a response.

Created SERVER-78430 as dependent on this ticket, to track the creation of a long-term and upstreamable fix for this problem.

Comment by Githook User [ 07/Jun/23 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-77694 Import immer lib from fork containing fix to always enable exceptions

(cherry picked from commit 016bb8a5ecc4b91e0a6fdcc58d0ff66c238fa970)
Branch: v7.0
https://github.com/mongodb/mongo/commit/a5abe1945467aafab256a3788bfe0fb25fa3f24c

Comment by Githook User [ 06/Jun/23 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-77694 Import immer lib from fork containing fix to always enable exceptions
Branch: master
https://github.com/mongodb/mongo/commit/016bb8a5ecc4b91e0a6fdcc58d0ff66c238fa970

Comment by Billy Donahue [ 01/Jun/23 ]

Opened bug with upstream just about the IMMER_RETHROW.
The exception detection is also broken and need another upstream ticket, I think.
https://github.com/arximboldi/immer/issues/264

Comment by Billy Donahue [ 01/Jun/23 ]

henrik.edin@mongodb.com feel free to reassign. I only named you because you'll know who owns this .

Comment by Billy Donahue [ 01/Jun/23 ]

Idea: maybe IMMER_RETHROW should be __builtin_unreachable() or similar if exceptions are disabled.

https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html#index-_005f_005fbuiltin_005funreachable

or into a call to any function marked [[noreturn]], such as std::terminate, which is a portable solution.

Comment by Billy Donahue [ 01/Jun/23 ]

https://mongodb.slack.com/archives/CQDQ9R413/p1685640114769829
Bottom line seems to be that there are two different immer bugs at work here.

  • incorrect detection of exception support, yielding failure to detect exception support in gcc.
  • IMMER_RETHROW looks like a function falling off the end without a return statement when exceptions are disabled.

As a result, it appears that the immer library will behave drastically differently on clang vs gcc. On clang builds it will be exception-based but on gcc builds the same code paths will kill the server.

Generated at Thu Feb 08 06:36:19 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.