Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-27675

tcmalloc must be explicitly linked by all targets in dynamic builds

XMLWordPrintableJSON

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 4.3.4
    • Affects Version/s: None
    • Component/s: Build, Internal Code
    • Labels:
      None
    • Fully Compatible
    • Platforms 2018-03-26, Dev Platform 2020-02-24

      A binary I made with dynamic linking consistently fails with:

      memory allocation/deallocation mismatch at 0x7ffff3569020: allocated with malloc being deallocated with delete []

      The link line for this binary was as follows:

      /opt/mongodbtoolchain/v2/bin/g++ -o build/A/mongo/bson/ugly_bson_integration_test -Wl,--no-as-needed -pthread -Wl,-z,now -rdynamic -Wl,--fatal-warnings -fstack-protector-strong -fuse-ld=gold -Wl,--build-id -Wl,--hash-style=gnu -Wl,--detect-odr-violations -Wl,-z,noexecstack -Wl,--warn-execstack -Wl,-z,relro -pie build/A/mongo/bson/ugly_bson_integration_test.o -Wl,-z,defs -Wl,--whole-archive -Wl,--start-group build/A/mongo/base/libsystem_error.so build/A/mongo/bson/mutable/libmutable_bson.so build/A/mongo/bson/util/libbson_extract.so build/A/mongo/client/libauthentication.so build/A/mongo/client/libclient_query.so build/A/mongo/client/libconnection_string.so build/A/mongo/client/libread_preference.so build/A/mongo/client/libsasl_client.so build/A/mongo/crypto/libcrypto_openssl.so build/A/mongo/crypto/libscramauth.so build/A/mongo/db/auth/libauth_rolename.so build/A/mongo/db/auth/libauthcommon.so build/A/mongo/db/commands/libtest_commands_enabled.so build/A/mongo/db/libcommon.so build/A/mongo/db/libdbmessage.so build/A/mongo/db/libindex_names.so build/A/mongo/db/libserver_options_core.so build/A/mongo/db/libserver_parameters.so build/A/mongo/db/libservice_context.so build/A/mongo/db/libwire_version.so build/A/mongo/db/libwrite_concern_options.so build/A/mongo/db/repl/liboptime.so build/A/mongo/db/stats/libcounters.so build/A/mongo/executor/libasync_stream.so build/A/mongo/executor/libasync_timer_asio.so build/A/mongo/executor/libconnection_pool.so build/A/mongo/executor/libconnection_pool_stats.so build/A/mongo/executor/libnetwork_interface.so build/A/mongo/executor/libnetwork_interface_asio.so build/A/mongo/executor/libnetwork_interface_asio_fixture.so build/A/mongo/executor/libnetwork_interface_factory.so build/A/mongo/executor/libnetwork_interface_thread_pool.so build/A/mongo/executor/libremote_command.so build/A/mongo/executor/libtask_executor_interface.so build/A/mongo/executor/libthread_pool_task_executor.so build/A/mongo/libbase.so build/A/mongo/logger/libmax_log_size.so build/A/mongo/rpc/libclient_metadata.so build/A/mongo/rpc/libcommand_reply.so build/A/mongo/rpc/libcommand_request.so build/A/mongo/rpc/libcommand_status.so build/A/mongo/rpc/libdocument_range.so build/A/mongo/rpc/liblegacy_reply.so build/A/mongo/rpc/liblegacy_request.so build/A/mongo/rpc/libmetadata.so build/A/mongo/rpc/libobject_check.so build/A/mongo/rpc/libprotocol.so build/A/mongo/rpc/librpc.so build/A/mongo/transport/libmessage_compressor.so build/A/mongo/transport/libtransport_layer_common.so build/A/mongo/transport/libtransport_layer_legacy.so build/A/mongo/transport/libtransport_layer_manager.so build/A/mongo/unittest/libintegration_test_main.so build/A/mongo/unittest/libunittest.so build/A/mongo/util/concurrency/libspin_lock.so build/A/mongo/util/concurrency/libthread_pool.so build/A/mongo/util/concurrency/libticketholder.so build/A/mongo/util/libbackground_job.so build/A/mongo/util/libclock_sources.so build/A/mongo/util/libdebugger.so build/A/mongo/util/libdecorable.so build/A/mongo/util/libfail_point.so build/A/mongo/util/libmd5.so build/A/mongo/util/libprocessinfo.so build/A/mongo/util/libquick_exit.so build/A/mongo/util/libsafe_num.so build/A/mongo/util/libversion_impl.so build/A/mongo/util/net/libhostandport.so build/A/mongo/util/net/libnetwork.so build/A/mongo/util/options_parser/liboptions_parser.so build/A/mongo/util/options_parser/liboptions_parser_init.so build/A/third_party/IntelRDFPMathLib20U1/libintel_decimal128.so build/A/third_party/asio-asio-1-11-0/libasio.so build/A/third_party/boost-1.60.0/libboost_chrono.so build/A/third_party/boost-1.60.0/libboost_filesystem.so build/A/third_party/boost-1.60.0/libboost_iostreams.so build/A/third_party/boost-1.60.0/libboost_program_options.so build/A/third_party/boost-1.60.0/libboost_regex.so build/A/third_party/boost-1.60.0/libboost_system.so build/A/third_party/boost-1.60.0/libboost_thread.so build/A/third_party/gperftools-2.5/libtcmalloc_minimal.so build/A/third_party/libshim_allocator.so build/A/third_party/libshim_asio.so build/A/third_party/libshim_boost.so build/A/third_party/libshim_intel_decimal128.so build/A/third_party/libshim_pcrecpp.so build/A/third_party/libshim_snappy.so build/A/third_party/libshim_tz.so build/A/third_party/libshim_yaml.so build/A/third_party/murmurhash3/libmurmurhash3.so build/A/third_party/pcre-8.39/libpcrecpp.so build/A/third_party/snappy-1.1.3/libsnappy.so build/A/third_party/yaml-cpp-0.5.3/libyaml.so -Wl,--end-group -Wl,--no-whole-archive -lm -lssl -lcrypto -lrt -ldl

      build/A/third_party/gperftools-2.5/libtcmalloc_minimal.so appears in the middle of the linkline.

      This error is being emitted during static initialization:

      #0  0x00007ffff3f9004f in raise () from /usr/lib/libc.so.6                                                                        [463/2278]
#1  0x00007ffff3f9147a in abort () from /usr/lib/libc.so.6
#2  0x00007ffff5c8a37e in LogPrintf (severity=-4, pat=0x7ffff5d2ec00 "memory allocation/deallocation mismatch at %p: allocated with %s bein$
 deallocated with %s", ap=0x7fffffffe1b0) at src/third_party/gperftools-2.5/src/base/logging.h:209
#3  0x00007ffff5c9add7 in RAW_LOG (lvl=-4, pat=0x7ffff5d2ec00 "memory allocation/deallocation mismatch at %p: allocated with %s being deall$
cated with %s") at src/third_party/gperftools-2.5/src/base/logging.h:228
#4  0x00007ffff5ca5c87 in MallocBlock::CheckLocked (this=0x7ffff3569000, type=-1125458062) at src/third_party/gperftools-2.5/src/debugalloc$
tion.cc:472
#5  0x00007ffff5ca58da in MallocBlock::CheckAndClear (this=0x7ffff3569000, type=-1125458062, given_size=0) at src/third_party/gperftools-2.$
/src/debugallocation.cc:410
#6  0x00007ffff5ca6025 in MallocBlock::Deallocate (this=0x7ffff3569000, type=-1125458062, given_size=0) at src/third_party/gperftools-2.5/s$
c/debugallocation.cc:564
#7  0x00007ffff5ca3005 in DebugDeallocate (ptr=0x7ffff3569020, type=-1125458062, given_size=0) at src/third_party/gperftools-2.5/src/debuga$
location.cc:1039
#8  0x00007ffff5d2a191 in tc_deletearray (p=0x7ffff3569020) at src/third_party/gperftools-2.5/src/debugallocation.cc:1350
#9  0x00007ffff774e7f6 in std::default_delete<mongo::UnorderedFastKeyTable<mongo::StringData, std::__cxx11::basic_string<char, std::char_tr$
its<char>, std::allocator<char> >, mongo::BSONObj::MatchType, mongo::StringMapTraits>::Entry []>::operator() (this=0x7fffffffe4a8, __ptr=0x$
ffff3569028) at /opt/mongodbtoolchain/v2/include/c++/5.4.0/bits/unique_ptr.h:119
#10 0x00007ffff774deb7 in std::unique_ptr<mongo::UnorderedFastKeyTable<mongo::StringData, std::__cxx11::basic_string<char, std::char_traits$
char>, std::allocator<char> >, mongo::BSONObj::MatchType, mongo::StringMapTraits>::Entry [], std::default_delete<mongo::UnorderedFastKeyTab$
e<mongo::StringData, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mongo::BSONObj::MatchType, mongo::Str$
ngMapTraits>::Entry []> >::~unique_ptr (
    this=0x7fffffffe4a8, __in_chrg=<optimized out>) at /opt/mongodbtoolchain/v2/include/c++/5.4.0/bits/unique_ptr.h:484
#11 0x00007ffff774d22e in mongo::UnorderedFastKeyTable<mongo::StringData, std::__cxx11::basic_string<char, std::char_traits<char>, std::all$
cator<char> >, mongo::BSONObj::MatchType, mongo::StringMapTraits>::Area::~Area (
    this=0x7fffffffe4a0, __in_chrg=<optimized out>) at src/mongo/util/unordered_fast_key_table.h:158
#12 0x00007ffff774f212 in mongo::UnorderedFastKeyTable<mongo::StringData, std::__cxx11::basic_string<char, std::char_traits<char>, std::all$
cator<char> >, mongo::BSONObj::MatchType, mongo::StringMapTraits>::_grow (
    this=0x7ffff77f71d0 <mongo::(anonymous namespace)::queryOperatorMap>) at src/mongo/util/unordered_fast_key_table_internal.h:192
#13 0x00007ffff774ec55 in mongo::UnorderedFastKeyTable<mongo::StringData, std::__cxx11::basic_string<char, std::char_traits<char>, std::all$
cator<char> >, mongo::BSONObj::MatchType, mongo::StringMapTraits>::try_emplace<>(mongo::StringMapTraits::HashedKey const&) (this=0x7ffff77f$
1d0 <mongo::(anonymous namespace)::queryOperatorMap>, key=...) at src/mongo/util/unordered_fast_key_table_internal.h:167
#14 0x00007ffff774dff8 in mongo::UnorderedFastKeyTable<mongo::StringData, std::__cxx11::basic_string<char, std::char_traits<char>, std::all$
cator<char> >, mongo::BSONObj::MatchType, mongo::StringMapTraits>::get (
    this=0x7ffff77f71d0 <mongo::(anonymous namespace)::queryOperatorMap>, key=...) at src/mongo/util/unordered_fast_key_table_internal.h:11$
#15 0x00007ffff774d3d8 in mongo::UnorderedFastKeyTable<mongo::StringData, std::__cxx11::basic_string<char, std::char_traits<char>, std::allo
cator<char> >, mongo::BSONObj::MatchType, mongo::StringMapTraits>::UnorderedFastKeyTable (
    this=0x7ffff77f71d0 <mongo::(anonymous namespace)::queryOperatorMap>, entries=...) at src/mongo/util/unordered_fast_key_table_internal..h
:103
#16 0x00007ffff7749d50 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at src/mongo/bson/bsonelement.cpp:3
10
#17 0x00007ffff7749dbe in _GLOBAL__sub_I_bsonelement.cpp(void) () at src/mongo/bson/bsonelement.cpp:1013
#18 0x00007ffff7de94fa in call_init.part () from /lib64/ld-linux-x86-64.so.2
#19 0x00007ffff7de960b in _dl_init () from /lib64/ld-linux-x86-64.so.2
#20 0x00007ffff7ddadaa in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#21 0x0000000000000001 in ?? ()
#22 0x00007fffffffebfa in ?? ()
#23 0x0000000000000000 in ?? ()

      This memory was allocated here:

      #0  MallocBlock::Initialize (this=0x7ffff3569000, size=776, type=-271733872) at src/third_party/gperftools-2.5/src/debugallocation.cc:395
#1  0x00007ffff5ca5f1c in MallocBlock::Allocate (size=776, type=-271733872) at src/third_party/gperftools-2.5/src/debugallocation.cc:543
#2  0x00007ffff5ca2e2b in DebugAllocate (size=776, type=-271733872) at src/third_party/gperftools-2.5/src/debugallocation.cc:1029
#3  0x00007ffff5ca70a0 in do_debug_malloc_or_debug_cpp_alloc (size=776) at src/third_party/gperftools-2.5/src/debugallocation.cc:1202
#4  0x00007ffff5d29808 in tc_malloc (size=776) at src/third_party/gperftools-2.5/src/debugallocation.cc:1219
#5  0x00007ffff7d882c8 in operator new (sz=776) at ../../../../gcc-5.4.0/libstdc++-v3/libsupc++/new_op.cc:50
#6  0x00007ffff7d88319 in operator new[] (sz=<optimized out>) at ../../../../gcc-5.4.0/libstdc++-v3/libsupc++/new_opv.cc:32
#7  0x00007ffff774f697 in mongo::UnorderedFastKeyTable<mongo::StringData, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mongo::BSONObj::MatchType, mongo::StringMapTraits>::Area::Area (
    this=0x7fffffffe4a0, capacity=16, maxProbe=1) at src/mongo/util/unordered_fast_key_table.h:164
#8  0x00007ffff774f1bd in mongo::UnorderedFastKeyTable<mongo::StringData, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mongo::BSONObj::MatchType, mongo::StringMapTraits>::_grow (
    this=0x7ffff77f71d0 <mongo::(anonymous namespace)::queryOperatorMap>) at src/mongo/util/unordered_fast_key_table_internal.h:186
#9  0x00007ffff774eb32 in mongo::UnorderedFastKeyTable<mongo::StringData, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mongo::BSONObj::MatchType, mongo::StringMapTraits>::try_emplace<>(mongo::StringMapTraits::HashedKey const&) (this=0x7ffff77f71d0 <mongo::(anonymous namespace)::queryOperatorMap>, key=...) at src/mongo/util/unordered_fast_key_table_internal.h:145
#10 0x00007ffff774dff8 in mongo::UnorderedFastKeyTable<mongo::StringData, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mongo::BSONObj::MatchType, mongo::StringMapTraits>::get (
    this=0x7ffff77f71d0 <mongo::(anonymous namespace)::queryOperatorMap>, key=...) at src/mongo/util/unordered_fast_key_table_internal.h:110
#11 0x00007ffff774d3d8 in mongo::UnorderedFastKeyTable<mongo::StringData, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mongo::BSONObj::MatchType, mongo::StringMapTraits>::UnorderedFastKeyTable (
    this=0x7ffff77f71d0 <mongo::(anonymous namespace)::queryOperatorMap>, entries=...) at src/mongo/util/unordered_fast_key_table_internal.h:103
#12 0x00007ffff7749d50 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at src/mongo/bson/bsonelement.cpp:310
#13 0x00007ffff7749dbe in _GLOBAL__sub_I_bsonelement.cpp(void) () at src/mongo/bson/bsonelement.cpp:1013
#14 0x00007ffff7de94fa in call_init.part () from /lib64/ld-linux-x86-64.so.2
#15 0x00007ffff7de960b in _dl_init () from /lib64/ld-linux-x86-64.so.2
#16 0x00007ffff7ddadaa in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#17 0x0000000000000001 in ?? ()
#18 0x00007fffffffebfa in ?? ()
#19 0x0000000000000000 in ?? ()

      Crucially, do_debug_malloc_or_debug_cpp_alloc calls

      void* p = DebugAllocate(size, MallocBlock::kMallocType);

      MallocBlock::kMallocType is saved into the allocated block as the alloc_type_. During deallocation, MallocBlock::kMallocType's AllocName, "free", is compared with the DeallocName of the freeing function. "free" is not the freeing function being used, so the deallocation explodes.

      This suggests that do_debug_malloc_or_debug_cpp_alloc is the wrong function to have been called. It's called by tc_malloc. There is another function called tc_newarray, which calls

      void* ptr = debug_cpp_alloc(size, MallocBlock::kArrayNewType, false);

      It has a DeallocName of

      (gdb) p DeallocName(MallocBlock::kArrayNewType)
      $68 = 0x7ffff5d2f31a "delete []"

      which suggests that was the correct allocation function to have been called.

            Assignee:
            andrew.morrow@mongodb.com Andrew Morrow (Inactive)
            Reporter:
            spencer.jackson@mongodb.com Spencer Jackson
            Votes:
            4 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: