[SERVER-48291] Avoid including a static copy of the GCC and C++ runtimes in every shared library Created: 19/May/20  Updated: 07/Nov/23  Resolved: 09/Aug/21

Status: Closed
Project: Core Server
Component/s: Build
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Andrew Morrow (Inactive) Assignee: Andrew Morrow (Inactive)
Resolution: Incomplete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
is duplicated by SERVER-49788 Investigate using LIBDEPS_PRIVATE for... Closed
Gantt Dependency
has to be done before SERVER-48281 Add a minimal MSAN variant Backlog
Problem/Incident
Related
related to SERVER-53304 Complete TODO listed in SERVER-48291 Closed
related to SERVER-53347 Cleanup for SERVER-48291 when merging... Closed
related to SERVER-82528 Complete TODO listed in SERVER-48291 Closed
related to SERVER-82894 Complete TODO listed in SERVER-48291 Closed
Sprint: Dev Platform 2020-08-24, Dev Platform 2020-09-07, Dev Platform 2020-09-21, Dev Platform 2020-10-05, Dev Platform 2020-10-19, Dev Platform 2020-11-02, Dev Platform 2020-11-16, Dev Platform 2020-11-30, Dev Platform 2020-12-14, Dev Platform 2021-05-17, Dev Platform 2021-05-31, Dev Platform 2021-06-14, Dev Platform 2021-07-12, Dev Platform 2021-07-26, Dev Platform 2021-08-09, Dev Platform 2021-08-23
Participants:
Linked BF Score: 45

 Description   

When building with --link-model=dynamic using the mongodb toolchain, every shared library is linked against a static copy of the GCC and C++ runtimes. This bloats the build, slows startup, and is just generally undesirable. We should find a way to avoid needing every shared library to embed a static copy of the GCC and runtimes when using the toolchain.



 Comments   
Comment by Githook User [ 28/Jul/21 ]

Author:

{'name': 'Andrew Morrow', 'email': 'acm@mongodb.com', 'username': 'acmorrow'}

Message: SERVER-48291 Remove need for individual third_party shim source files

(cherry picked from commit 0c1e060e7e87a32673e9a6b0311663aa4875544b)
Branch: v5.0
https://github.com/mongodb/mongo/commit/06b7cb6d97d4c2425f7cfcd070be23ed34e00b3e

Comment by Githook User [ 28/Jul/21 ]

Author:

{'name': 'Andrew Morrow', 'email': 'acm@mongodb.com', 'username': 'acmorrow'}

Message: SERVER-48291 Remove support for shim_cxx and shim_crt libraries

(cherry picked from commit 787d5a2aee7d69572345e0b33447a8b2c92ea982)
Branch: v5.0
https://github.com/mongodb/mongo/commit/2a3ac32087537aed015c159de7092a103a14bd9d

Comment by Githook User [ 27/Jul/21 ]

Author:

{'name': 'Andrew Morrow', 'email': 'acm@mongodb.com', 'username': 'acmorrow'}

Message: SERVER-48291 Remove need for individual third_party shim source files
Branch: master
https://github.com/mongodb/mongo/commit/0c1e060e7e87a32673e9a6b0311663aa4875544b

Comment by Githook User [ 26/Jul/21 ]

Author:

{'name': 'Andrew Morrow', 'email': 'acm@mongodb.com', 'username': 'acmorrow'}

Message: SERVER-48291 Remove support for shim_cxx and shim_crt libraries
Branch: master
https://github.com/mongodb/mongo/commit/787d5a2aee7d69572345e0b33447a8b2c92ea982

Comment by Githook User [ 20/Jul/21 ]

Author:

{'name': 'Andrew Morrow', 'email': 'acm@mongodb.com', 'username': 'acmorrow'}

Message: SERVER-48291 Restore design of the boost assert shim

This was inadvertently lumped in with the other ShimLibrary
conversions, but it doesn't actually follow the model. This
library has real symbols.

(cherry picked from commit 1c41fa28a3ce3a5e661e951aa0d435de6815797d)
Branch: v5.0
https://github.com/mongodb/mongo/commit/5883d5d2cfb19ca8df2b03d9b49c15db66c36319

Comment by Githook User [ 06/Jul/21 ]

Author:

{'name': 'Andrew Morrow', 'email': 'acm@mongodb.com', 'username': 'acmorrow'}

Message: SERVER-48291 Restore design of the boost assert shim

This was inadvertently lumped in with the other ShimLibrary
conversions, but it doesn't actually follow the model. This
library has real symbols.
Branch: master
https://github.com/mongodb/mongo/commit/1c41fa28a3ce3a5e661e951aa0d435de6815797d

Comment by Githook User [ 09/Dec/20 ]

Author:

{'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}

Message: SERVER-48291 Ensure runtime is dynamically linked in dynamic builds

Prior to this point, a dynamic build might have resulted in some runtime
libraries being statically linked into shared objects and executables in
cases where "shared" runtime libraries were actually linker scripts that
linked static versions. This was the case with the MongoDB toolchain and
some distro toolchains, including those installed as updated compiler
versions in RHEL.

The effect of having runtime libraries statically linked was that
symbols from those libraries would end up scattered over the compiled
objects, increasing object sizes and slowing down server startup. Now,
whenever a dynamic build is selected, the user can choose whether to
create "shim" runtime libraries that wrap the static ones.

The default behavior remains as it was before, and dynamic runtime must
be enabled in order to use it.
Branch: master
https://github.com/mongodb/mongo/commit/b0120ee8b2a50a62f0b92a259f32d82f4c1aea22

Comment by Githook User [ 26/Nov/20 ]

Author:

{'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}

Message: Revert "SERVER-48291 Ensure runtime is dynamically linked in dynamic builds"

This reverts commit 2a0e76082be0f2aca82830bcaf91f6d737b842ac.
Branch: master
https://github.com/mongodb/mongo/commit/710aa34bd1bb9ff1c13a41737c86e636e57f0a88

Comment by Githook User [ 26/Nov/20 ]

Author:

{'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}

Message: SERVER-48291 Ensure runtime is dynamically linked in dynamic builds

Prior to this point, a dynamic build might have resulted in some runtime
libraries being statically linked into shared objects and executables in
cases where "shared" runtime libraries were actually linker scripts that
linked static versions. This was the case with the MongoDB toolchain and
some distro toolchains, including those installed as updated compiler
versions in RHEL.

The effect of having runtime libraries statically linked was that
symbols from those libraries would end up scattered over the compiled
objects, increasing object sizes and slowing down server startup. Now,
whenever a dynamic build is selected, the user can choose whether to
create "shim" runtime libraries that wrap the static ones.

The default behavior on Linux is that dynamic builds will detect whether
runtime libraries are linker scripts and create shim libraries if any are
found. On Windows, the default is to always use a dynamic runtime library
with dynamic builds. For other platforms, the prior behavior remains
unchanged.
Branch: master
https://github.com/mongodb/mongo/commit/2a0e76082be0f2aca82830bcaf91f6d737b842ac

Comment by Githook User [ 26/Nov/20 ]

Author:

{'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}

Message: SERVER-48291 Add global dependency pushdown to libdeps

We sometimes have situations where a dependency applies at a large
scope, such as in the case of tcmalloc, which can apply everywhere. What
we have done previously is to hack these dependencies into the LIBDEPS
environment variable by adding a builder to all nodes that can produce a
compiler result. This is, as stated previously, hackish and hard to
control, and it results in adding a Public dependency to all those
nodes.

What we now do instead is to define LIBDEPS_GLOBAL on the *build
environment* (not the Builder node) listing the targets we would like to
push down to all other nodes below that point. This has the effect of
adding those targets as Private dependencies on all Builder nodes from
that point downward, which means some common Public dependencies can be
converted to a Private dependency that is stated only once.
Branch: master
https://github.com/mongodb/mongo/commit/82ef2151223ec184682e752436ac07916500253f

Comment by Githook User [ 26/Nov/20 ]

Author:

{'name': 'Ryan Egesdahl', 'email': 'ryan.egesdahl@mongodb.com', 'username': 'deriamis'}

Message: SERVER-48291 Add global dependency pushdown to libdeps
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/a1fcf705564c2efb8a42791bd856b053270ea8ba

Generated at Thu Feb 08 05:16:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.