[SERVER-21207] Leak in AsyncStream::connect Created: 29/Oct/15  Updated: 14/Apr/16  Resolved: 29/Oct/15

Status: Closed
Project: Core Server
Component/s: Networking
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Robert Guo (Inactive) Assignee: Robert Guo (Inactive)
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: TIG C (11/20/15)
Participants:

 Description   

Triggered by the fuzzer. Did not repro when building with gcc 4.8
mongo git hash: 8c09883af1

[ShardedClusterFixture:job0:shard0] Direct leak of 64 byte(s) in 4 object(s) allocated from:
[ShardedClusterFixture:job0:shard0]     #0 0xae13c9 in operator new(unsigned long) (/home/ubuntu/mongo/mongod+0xae13c9)
[ShardedClusterFixture:job0:shard0]     #1 0x27a3967 in std::_Function_base::_Base_manager<mongo::executor::NetworkInterfaceASIO::_setupSocket(mongo::executor::NetworkInterfaceASIO::AsyncOp*, asio::ip::basic_resolver_iterator<asio::ip::tcp>)::$_1>::_M_clone(std::_Any_data&, std::_Any_data const&, std::integral_constant<bool, false>) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1910
[ShardedClusterFixture:job0:shard0]     #2 0x27a3967 in std::_Function_base::_Base_manager<mongo::executor::NetworkInterfaceASIO::_setupSocket(mongo::executor::NetworkInterfaceASIO::AsyncOp*, asio::ip::basic_resolver_iterator<asio::ip::tcp>)::$_1>::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1946
[ShardedClusterFixture:job0:shard0]     #3 0x270fe69 in basic_resolver_iterator /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:2442
[ShardedClusterFixture:job0:shard0]     #4 0x270fe69 in mongo::executor::AsyncStream::connect(asio::ip::basic_resolver_iterator<asio::ip::tcp>, std::function<void (std::error_code)>&&) /home/ubuntu/mongo/src/mongo/executor/async_stream.cpp:50
[ShardedClusterFixture:job0:shard0]     #5 0x27a1500 in mongo::executor::NetworkInterfaceASIO::_setupSocket(mongo::executor::NetworkInterfaceASIO::AsyncOp*, asio::ip::basic_resolver_iterator<asio::ip::tcp>) /home/ubuntu/mongo/src/mongo/executor/network_interface_asio_connect.cpp:117
[ShardedClusterFixture:job0:shard0]     #6 0x27a6251 in operator() /home/ubuntu/mongo/src/mongo/executor/network_interface_asio_connect.cpp:103
[ShardedClusterFixture:job0:shard0]     #7 0x27a6251 in basic_resolver_iterator /home/ubuntu/mongo/src/mongo/executor/network_interface_asio.h:348
[ShardedClusterFixture:job0:shard0]     #8 0x27a6251 in basic_resolver_iterator /home/ubuntu/mongo/src/mongo/executor/network_interface_asio_connect.cpp:102
[ShardedClusterFixture:job0:shard0]     #9 0x27a6251 in asio::detail::binder2<mongo::executor::NetworkInterfaceASIO::_connect(mongo::executor::NetworkInterfaceASIO::AsyncOp*)::$_0, std::error_code, asio::ip::basic_resolver_iterator<asio::ip::tcp> >::operator()() /home/ubuntu/mongo/src/third_party/asio-asio-1-11-0/asio/include/asio/detail/bind_handler.hpp:160
[ShardedClusterFixture:job0:shard0]     #10 0x27a6251 in void asio::asio_handler_invoke<asio::detail::binder2<mongo::executor::NetworkInterfaceASIO::_connect(mongo::executor::NetworkInterfaceASIO::AsyncOp*)::$_0, std::error_code, asio::ip::basic_resolver_iterator<asio::ip::tcp> > >(asio::detail::binder2<mongo::executor::NetworkInterfaceASIO::_connect(mongo::executor::NetworkInterfaceASIO::AsyncOp*)::$_0, std::error_code, asio::ip::basic_resolver_iterator<asio::ip::tcp> >&, ...) /home/ubuntu/mongo/src/third_party/asio-asio-1-11-0/asio/include/asio/handler_invoke_hook.hpp:68
[ShardedClusterFixture:job0:shard0]     #11 0x27a6251 in void asio_handler_invoke_helpers::invoke<asio::detail::binder2<mongo::executor::NetworkInterfaceASIO::_connect(mongo::executor::NetworkInterfaceASIO::AsyncOp*)::$_0, std::error_code, asio::ip::basic_resolver_iterator<asio::ip::tcp> >, mongo::executor::NetworkInterfaceASIO::_connect(mongo::executor::NetworkInterfaceASIO::AsyncOp*)::$_0>(asio::detail::binder2<mongo::executor::NetworkInterfaceASIO::_connect(mongo::executor::NetworkInterfaceASIO::AsyncOp*)::$_0, std::error_code, asio::ip::basic_resolver_iterator<asio::ip::tcp> >&, mongo::executor::NetworkInterfaceASIO::_connect(mongo::executor::NetworkInterfaceASIO::AsyncOp*)::$_0&) /home/ubuntu/mongo/src/third_party/asio-asio-1-11-0/asio/include/asio/detail/handler_invoke_helpers.hpp:37
[ShardedClusterFixture:job0:shard0]     #12 0x27a6251 in void asio::detail::handler_work<mongo::executor::NetworkInterfaceASIO::_connect(mongo::executor::NetworkInterfaceASIO::AsyncOp*)::$_0, asio::system_executor>::complete<asio::detail::binder2<mongo::executor::NetworkInterfaceASIO::_connect(mongo::executor::NetworkInterfaceASIO::AsyncOp*)::$_0, std::error_code, asio::ip::basic_resolver_iterator<asio::ip::tcp> > >(asio::detail::binder2<mongo::executor::NetworkInterfaceASIO::_connect(mongo::executor::NetworkInterfaceASIO::AsyncOp*)::$_0, std::error_code, asio::ip::basic_resolver_iterator<asio::ip::tcp> >&, mongo::executor::NetworkInterfaceASIO::_connect(mongo::executor::NetworkInterfaceASIO::AsyncOp*)::$_0&) /home/ubuntu/mongo/src/third_party/asio-asio-1-11-0/asio/include/asio/detail/handler_work.hpp:81
[ShardedClusterFixture:job0:shard0]     #13 0x27a6251 in asio::detail::resolve_op<asio::ip::tcp, mongo::executor::NetworkInterfaceASIO::_connect(mongo::executor::NetworkInterfaceASIO::AsyncOp*)::$_0>::do_complete(void*, asio::detail::scheduler_operation*, std::error_code const&, unsigned long) /home/ubuntu/mongo/src/third_party/asio-asio-1-11-0/asio/include/asio/detail/resolve_op.hpp:113
[ShardedClusterFixture:job0:shard0]     #14 0x30d709f in asio::detail::scheduler_operation::complete(void*, std::error_code const&, unsigned long) /home/ubuntu/mongo/src/third_party/asio-asio-1-11-0/asio/include/asio/detail/scheduler_operation.hpp:39
[ShardedClusterFixture:job0:shard0]     #15 0x30d709f in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock<asio::detail::posix_mutex>&, asio::detail::scheduler_thread_info&, std::error_code const&) /home/ubuntu/mongo/src/third_party/asio-asio-1-11-0/asio/include/asio/detail/impl/scheduler.ipp:369
[ShardedClusterFixture:job0:shard0]     #16 0x30bb859 in asio::detail::scheduler::run(std::error_code&) /home/ubuntu/mongo/src/third_party/asio-asio-1-11-0/asio/include/asio/detail/impl/scheduler.ipp:146
[ShardedClusterFixture:job0:shard0]     #17 0x30bb4df in asio::io_service::run() /home/ubuntu/mongo/src/third_party/asio-asio-1-11-0/asio/include/asio/impl/io_service.ipp:60
[ShardedClusterFixture:job0:shard0]     #18 0x276e93c in operator() /home/ubuntu/mongo/src/mongo/executor/network_interface_asio.cpp:107
[ShardedClusterFixture:job0:shard0]     #19 0x276e93c in void std::_Bind_simple<mongo::executor::NetworkInterfaceASIO::startup()::$_0 ()>::_M_invoke<>(std::_Index_tuple<>) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1731
[ShardedClusterFixture:job0:shard0]     #20 0x276e93c in std::_Bind_simple<mongo::executor::NetworkInterfaceASIO::startup()::$_0 ()>::operator()() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/functional:1720
[ShardedClusterFixture:job0:shard0]     #21 0x276e93c in std::thread::_Impl<std::_Bind_simple<mongo::executor::NetworkInterfaceASIO::startup()::$_0 ()> >::_M_run() /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/thread:115
[ShardedClusterFixture:job0:shard0]     #22 0x7f52a39eda3f (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xb1a3f)



 Comments   
Comment by Robert Guo (Inactive) [ 29/Oct/15 ]

@Drew This isn't a problem for our ASAN builders since they have been updated to Ubuntu 1410 and clang 3.5. Our clang builds on mongodb.org are also using clang-3.5.
See: https://github.com/mongodb/mongo/commit/0924e369fb45d6597e3ad72c701fca9be7c61b32#diff-71ccc9b828b2d68dab47c8be07ab6f96R3663

@Adam I looked on llvm and gcc's bugzilla but didn't find anything relevant. Currently I don't think this bug has any real impact. If it turns up in an Evergreen ASAN builder in the future (when jstestfuzzer + sharding is committed), I can dig a little deeper.

my search on gcc: https://gcc.gnu.org/bugzilla/buglist.cgi?bug_status=RESOLVED&bug_status=VERIFIED&bug_status=CLOSED&cf_known_to_fail=4.8.0&cf_known_to_fail=4.8.1&cf_known_to_fail=4.8.2&cf_known_to_fail=4.8.3&cf_known_to_fail=4.8.4&cf_known_to_fail=4.8.5&cf_known_to_fail_type=allwords&cf_known_to_work_type=allwords&component=libstdc%2B%2B&query_format=advanced&short_desc=leak&short_desc_type=allwords

Comment by Andrew Morrow (Inactive) [ 29/Oct/15 ]

I second Adam's concern. If we don't address this, we are going to have red tests on the ASAN builder and that is a big problem as it can mask other errors. Potentially, we could upgrade the ASAN builder to use a newer OS image that comes with newer clang to work around?

Comment by Adam Midvidy [ 29/Oct/15 ]

robert.guo is there a relevant libstdc++ or clang bug for this? I want to make sure that it is fixed in the toolchains with which we build our releases.

Comment by Robert Guo (Inactive) [ 29/Oct/15 ]

This is most likely caused by a leak in libstdc++; it occurs frequently on Ubuntu 14.04.3 with libstdc++ 6.0.19 and clang 3.4.

Using clang 3.5 or 3.6 fixes the problem.

I will close this ticket as a "Won't fix" as it is not an issue with MongoDB's code

Generated at Thu Feb 08 03:56:39 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.