[SERVER-41835] symbol clashes: libgcc vs vendored IntelRDFPMathLib20U1 Created: 20/Jun/19  Updated: 28/Aug/23

Status: Backlog
Project: Core Server
Component/s: Internal Code
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Billy Donahue Assignee: Backlog - Service Architecture
Resolution: Unresolved Votes: 0
Labels: toolchain-next
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-48546 Linker error building with scons --gd... Closed
related to SERVER-49375 Clang with --link-model=dynamic produ... Closed
related to SERVER-24374 Decimal128 constructors taking intege... Closed
Assigned Teams:
Service Arch
Participants:

 Description   

https://jira.mongodb.org/browse/SERVER-24374?focusedCommentId=2287679&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-2287679

Ran into a not-quite-diagnosed issue with the LIBDEPS_PRIVATE change in the original attempt at SERVER-24374. It means that the system's libgcc, which provides its own intelfp names, can seep into dynamic builds of the decimal128_test.

We're doing something wrong. For example, here's a grep for _bid128_to_int32_rnint, among the 'nm' output for each dependency in the 'ldd' output from decimal128_test,
when built with dynamic gcc.

build/dynamic_gcc/mongo/unittest/libunittest_main.so:                      T __bid128_to_int32_rnint
build/dynamic_gcc/mongo/db/libserver_options_core.so:                      T __bid128_to_int32_rnint
build/dynamic_gcc/third_party/IntelRDFPMathLib20U1/libintel_decimal128.so: T __bid128_to_int32_rnint
build/dynamic_gcc/mongo/util/net/libhttp_client.so:                        T __bid128_to_int32_rnint
build/dynamic_gcc/mongo/util/libsecure_compare_memory.so:                  T __bid128_to_int32_rnint

I suspect that the __ (double-underscore) prefix of the libgcc and vendored names is a configurable parameter of the library. If so, our vendored copy of the library should use another prefix. Double underscores are disallowed in identifiers emitted by application-level libraries, so we breaking the C rules by having a vendored library that emits them.

This restriction seems a little academic, but here we are with a real problem caused by the violation.



 Comments   
Comment by Billy Donahue [ 21/Jun/19 ]

After some experimentation, the bid_conf.h is insufficient to rename all the names exported by the library.
There are a few hand edits necessary, and then float128/ subdir is completely untouched.

We should be reluctant to do too much customization inside the library. Need another approach.

Some insights into the underlying problem have emerged in the investigation.

LIBDEPS_PRIVATE change in the original attempt at SERVER-24374. It means that the system's libgcc, which provides its own intelfp names, can seep into dynamic builds of the decimal128_test.

By "system's libgcc" above, that really means the toolchain's "-lgcc_s",
which resolves to:

$ cat /opt/mongodbtoolchain/v3/lib/gcc/x86_64-mongodb-linux/8.2.0/libgcc_s.so 
/* GNU ld script
Use the shared library, but fall back to static library. */
OUTPUT_FORMAT(elf64-x86-64)
GROUP ( /lib/x86_64-linux-gnu/libgcc_s.so.1 libgcc.a )

And so libgcc.a is statically linked into each

mongo/**/lib*.so

target.

When "base" has a LIBDEPS_PRIVATE dep on intelfp, then the static link to libgcc.a's bid* functions happen in each .so and we can't overcome it with later dynamic linking.

When "base" instead has a LIBDEPS dep, then the vendored "intel_decimal128" library appears on the link line of each .so, so it is interposed before the "fake" libgcc_s.so linker script (which would otherwise provide libgcc.a's intelfp functions).

So because toolchain is using a linker script to force static linking of libgcc_s names, we can't override them using LIBDEPS_PRIVATE. We MUST use public LIBDEPS.

Comment by Billy Donahue [ 20/Jun/19 ]

Yes, the emitted names editable in a bid_conf.h header.

https://github.com/mongodb/mongo/blob/master/src/third_party/IntelRDFPMathLib20U1/README#L94-L100

  3. FUNCTION NAMES
  =================
 
  The function names used in the library are not identical to the names from
the IEEE Standard 754-2008 for Floating-Point Arithmetic. The mapping between
the two sets is given in Section 10 below. The function names can be changed
by editing the #define statements at the beginning of bid_conf.h.

https://github.com/mongodb/mongo/blob/master/src/third_party/IntelRDFPMathLib20U1/LIBRARY/src/bid_conf.h#L41-L817

That's a lot of stuff to edit. It would be cute if they left a prefixing system, but maybe we can set one up.

Generated at Thu Feb 08 04:58:48 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.