[SERVER-49141] Coverity build of mongodb/mongo failing with link error for libunwind Created: 26/Jun/20  Updated: 29/Oct/23  Resolved: 29/Jun/20

Status: Closed
Project: Core Server
Component/s: Build
Affects Version/s: None
Fix Version/s: 4.7.0

Type: Bug Priority: Major - P3
Reporter: Eric Milkie Assignee: Andrew Morrow (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-49074 Change unittest pcrecpp dep from LIBD... Closed
Related
is related to SERVER-49148 Implement a facility for safer enviro... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Dev Platform 2020-06-29, Dev Platform 2020-07-13
Participants:
Linked BF Score: 0

 Description   

Coverity builds with this command line:

/usr/local/bin/python3 buildscripts/scons.py --install-mode=hygienic install-all --keep-going --variables-files=etc/scons/mongodbtoolchain_stable_gcc.vars --disable-warnings-as-errors -j6 --opt=off --dbg=off --allocator=system --link-model=dynamic

 

And as of last night, ran a build at the head of mongodb main branch and encountered this linker error:

 
Using tempfile /data/tmp/tmp8igx5fge.lnk for command line:
/opt/mongodbtoolchain/v3/bin/g++ -o build/7a67446c/mongo/util/stacktrace_bm -Wl,--no-as-needed -pthread -Wl,-z,now -rdynamic -fstack-protector-strong -fuse-ld=gold -Wl,--gdb-index -Wl,--no-threads -Wl,--build-id -Wl,--hash-style=gnu -Wl,-z,noexecstack -Wl,--warn-execstack -Wl,-z,relro -Wl,-rpath,/usr/lib64/perl5/CORE -Wl,-z,origin -Wl,--enable-new-dtags -pie -Wl,-rpath=\$ORIGIN/../lib build/7a67446c/mongo/util/stacktrace_bm.o -Wl,-z,defs -Wl,--whole-archive build/7a67446c/mongo/unittest/libbenchmark_main.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/libshim_benchmark.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/benchmark/libbenchmark.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/db/libservice_context.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/util/net/libnetwork.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/util/libperiodic_runner.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/util/libfail_point.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/util/libclock_sources.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/util/concurrency/libspin_lock.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/db/storage/libwrite_unit_of_work.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/db/storage/librecovery_unit_base.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/db/libwrite_concern_options.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/db/libread_write_concern_provenance.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/bson/util/libbson_extract.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/db/libmulti_key_path_tracker.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/db/liblogical_session_id.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/idl/libserver_parameter.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/idl/libidl_parser.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/db/libcommand_generic_argument.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/db/commands/libserver_status_core.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/crypto/libsha_block_openssl.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/crypto/libsha1_block.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/crypto/libsha256_block.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/util/libsecure_compare_memory.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/libbase.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/murmurhash3/libmurmurhash3.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/libshim_fmt.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/fmt/libfmt.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/libshim_boost.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/boost-1.70.0/libboost_program_options.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/boost-1.70.0/libboost_log.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/boost-1.70.0/libboost_thread.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/boost-1.70.0/libboost_iostreams.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/boost-1.70.0/libboost_filesystem.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/boost-1.70.0/libboost_system.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/mongo/libboost_assert_shim.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/libshim_abseil.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/abseil-cpp-master/libabsl_container.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/abseil-cpp-master/libabsl_throw_delegate.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/abseil-cpp-master/libabsl_hash.so -Wl,--no-whole-archive -Wl,--whole-archive build/7a67446c/third_party/libshim_allocator.so -Wl,--no-whole-archive -llzma -llzma -llzma -lm -lresolv -lcrypto -lssl -lcrypto -ldl -lrt -ldl
/opt/mongodbtoolchain/v3/bin/g++ @/data/tmp/tmp8igx5fge.lnk
src/mongo/util/stacktrace_bm.cpp:113: error: undefined reference to '_Ux86_64_getcontext'
src/mongo/util/stacktrace_bm.cpp:114: error: undefined reference to '_Ux86_64_strerror'
src/mongo/util/stacktrace_bm.cpp:118: error: undefined reference to '_ULx86_64_init_local'
src/mongo/util/stacktrace_bm.cpp:119: error: undefined reference to '_Ux86_64_strerror'
src/mongo/util/stacktrace_bm.cpp:124: error: undefined reference to '_ULx86_64_step'
src/mongo/util/stacktrace_bm.cpp:125: error: undefined reference to '_Ux86_64_strerror'
collect2: error: ld returned 1 exit status
scons: *** [build/7a67446c/mongo/util/stacktrace_bm] Error 1 



 Comments   
Comment by Githook User [ 29/Jun/20 ]

Author:

{'name': 'Andrew Morrow', 'email': 'acm@mongodb.com', 'username': 'acmorrow'}

Message: SERVER-49141 Ensure that LIBDEPS for libunwind do not accidentally become public
Branch: master
https://github.com/mongodb/mongo/commit/5394c832f05414fabd069c54066746e127f5d2e0

Comment by Andrew Morrow (Inactive) [ 26/Jun/20 ]

This was caused by the changes in SERVER-48546. The root cause is interesting enough to warrant a somewhat detailed writeup.

At its simplest, the cause here is that when building in --link-model=dynamic --allocator=system the stacktrace_bm target doesn't end up with libunwind on its link line, as it clearly must always have if it is going to link, since it directly uses symbols from the library.

A quick look suggests that the answer is simple: SERVER-48546 (apparently, read on) moved shim_unwind from LIBDEPS to LIBDEPS_PRIVATE for base and libtcmalloc_minimal, so in a --link-model=dynamic build, the stacktrace_bm Program was no longer picking up a transitive dependency to shim_unwind, which would cause it to fail to link. So the answer appears to be that shim_unwind should be added to LIBDEPS for stacktrace_bm.

This leads to an obvious question of why such a clear error didn't fail either the ! Shared Library Enterprise RHEL 6.2 builders compile_all task, or the Enterprise RHEL 6.2 (Benchmarks) builders compile_benchmarks task. The latter is easily answered: the Benchmarks builder builds in static mode, not dynamic. It also implicitly uses --allocator=tcmalloc, and ultimately the tcmalloc_minimal library has a dependency on libunwind. Since the PUBLIC/PRIVATE distinction doesn't exist in static builds, the link of stacktrace_bm includes libunwind via transitivity, and everything works.

The former case is much more interesting. If libunwind has been made LIBDEPS_PRIVATE to both base and tcmalloc_minimal, how could stacktrace_bm have linked at all? Since we are using --link-model=dynamic, LIBDEPS_PRIVATE is enabled, and libunwind should not appear on the link line for stacktrace_bm, so we would expect the link to fail. But when we examine the logs, we find that, somehow libunwind does appear on the link line for stacktrace_bm. So everything links just fine.

The important clue is that if you switch the build to use --allocator=system --link-model=dynamic, suddenly the test breaks. So, despite the fact that after SERVER-48546 libtcmalloc_minimal declares libunwind as a LIBDEPS_PRIVATE edge, it appears to be behaving as if it was a public LIBDEP. How can this be?

Searching for all instances of shim_unwind provides the answer. The behavior of env.InjectThirdParty(libraries=['unwind']), which is used in the Environment setup for both base and tcmalloc_minimal sets LIBDEPS to ['$BUILD_DIR/third_party/shim_unwind'], effectively making the unwind dependency a public LIBDEP in the affected Environment.

But if that were the whole story, then shim_unwind would have become a public LIBDEP for libbase and stacktrace_bm should have linked!

Which brings in the last piece. The Library declaration for base has explicit LIBDEPS and LIBDEPS_PRIVATE overrides in its environment. So the LIBDEPS value set by env.InjectThirdParty(libraries=['unwind']) is overwritten and ignored. So unwind remains a private LIBDEP of base: linking only to base won't get you unwind. But the Library declaration for tcmalloc_minimal only has an override for LIBDEPS_PRIVATE, so the LIBDEPS value injected by env.InjectThirdParty(libraries=['unwind']), remains effective: unwind is LIBDEPS_PUBLIC in tcmalloc_minimal (and also LIBDEPS_PRIVATE, which is why SERVER-48638 would have caught this). So, as long as -allocator=tcmalloc is in play, stacktrace_bm picks up a dependency on libunwind transitively via tcmalloc_minimal. But if you switch to -allocator=system, then suddenly you don't pick up that tricky transitive dependency, and unwind is now private to libbase, and stacktrace_bm doesn't declare a dependency, so it fails to link.

There are several lessons to learn from this:

  • We should implement the suggestions in SERVER-48638, which would have flagged that shim_libunwind was both public and private to libtcmalloc_minimal.
  • We should remove the injection into LIBDEPS in env.InjectThirdParty(libraries=['unwind']). Stylistically, libraries don't expect the env to already have LIBDEPS[_PRIVATE|INTERFACE] set and always do a hard override.
  • We should investigate a less fragile mechanism for InjectThirdParty which does not allow its modifications to be silently discarded.

The immediate fix for the issue is straightforward. We will remove the modification of LIBDEPS from the behavior of env.InjectThirdParty(libraries=['unwind']), and directly attach shim_unwind in all the required places.

Generated at Thu Feb 08 05:19:02 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.