[CXX-445] SIGABRT on shutdown with SSL enabled Created: 10/Dec/14  Updated: 19/Dec/14  Resolved: 11/Dec/14

Status: Closed
Project: C++ Driver
Component/s: Documentation
Affects Version/s: legacy-1.0.0-rc2
Fix Version/s: legacy-1.0.0-rc3

Type: Improvement Priority: Major - P3
Reporter: Pieter Jordaan Assignee: Andrew Morrow (Inactive)
Resolution: Done Votes: 0
Labels: legacy-cxx
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Here is a stacktrace

terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::lock_error> >'
  what():  boost: mutex lock failed in pthread_mutex_lock: Invalid argument
 
Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff272f700 (LWP 27089)]
0x00007ffff598ad27 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56      ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) thread apply all backtrace
 
Thread 3 (Thread 0x7ffff272f700 (LWP 27089)):
#0  0x00007ffff598ad27 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff598c418 in __GI_abort () at abort.c:89
#2  0x00007ffff5f9105d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff5f8eed6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff5f8ef21 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff5f8f139 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007ffff76c51af in boost::throw_exception<boost::lock_error> (e=...) at /usr/local/include/boost/throw_exception.hpp:69
#7  0x00007ffff77448ea in lock (this=0x7ffff799d120 <mongo::(anonymous namespace)::sslManagerMtx>) at /usr/local/include/boost/thread/pthread/mutex.hpp:119
#8  lock_guard (m_=..., this=<synthetic pointer>) at /usr/local/include/boost/thread/lock_guard.hpp:38
#9  mongo::getSSLManager () at src/mongo/util/net/ssl_manager.cpp:316
#10 0x00007ffff772ade6 in mongo::BackgroundJob::jobBody (this=0x7be4d0) at src/mongo/util/background.cpp:83
#11 0x00007ffff66ccc1a in thread_proxy () from /usr/local/lib/libboost_thread.so.1.57.0
#12 0x00007ffff7bc40a5 in start_thread (arg=0x7ffff272f700) at pthread_create.c:309
#13 0x00007ffff5a4e77d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
 
Thread 1 (Thread 0x7ffff7fb0780 (LWP 27082)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007ffff772b60b in wait (m=..., this=0x7bf518) at /usr/local/include/boost/thread/pthread/condition_variable.hpp:73
#2  mongo::BackgroundJob::wait (this=this@entry=0x7be4d0, msTimeOut=msTimeOut@entry=0) at src/mongo/util/background.cpp:154
#3  0x00007ffff76f5742 in ~ReplicaSetMonitorWatcher (this=0x7be4d0, __in_chrg=<optimized out>) at src/mongo/client/replica_set_monitor.cpp:107
#4  ~ReplicaSetMonitorWatcher (this=0x7be4d0, __in_chrg=<optimized out>) at src/mongo/client/replica_set_monitor.cpp:109
#5  checked_delete<mongo::(anonymous namespace)::ReplicaSetMonitorWatcher> (x=0x7be4d0) at /usr/local/include/boost/core/checked_delete.hpp:34
#6  boost::scoped_ptr<mongo::(anonymous namespace)::ReplicaSetMonitorWatcher>::~scoped_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /usr/local/include/boost/smart_pt
#7  0x00007ffff598e92a in __cxa_finalize (d=0x7ffff799c000) at cxa_finalize.c:56
#8  0x00007ffff7672773 in __do_global_dtors_aux () from /usr/local/lib/libmongoclient.so
#9  0x00007fffffffdb50 in ?? ()
#10 0x00007ffff7dea73a in _dl_fini () at dl-fini.c:252
Backtrace stopped: frame did not save the PC

Built with --sharedclient --ssl --c++11
Boost version 1.57

Please let me know how I can assist in debugging this.



 Comments   
Comment by Pieter Willem Jordaan [ 11/Dec/14 ]

Thanks so much Andrew. I'm stoked about the support MongoDB provides

Comment by Andrew Morrow (Inactive) [ 11/Dec/14 ]

Hi pieterwjordaanpc@gmail.com -

I've updated the documentation on the wiki to include more prominent links to information on configuring the driver, and updated the configuration documentation to reflect the current state of the code.

Comment by Andrew Morrow (Inactive) [ 10/Dec/14 ]

The documentation describing initialization and termination (https://github.com/mongodb/mongo-cxx-driver/wiki/Configuring-the-Legacy-Driver) is out of date and should be updated to reflect currently reality.

Comment by Andrew Morrow (Inactive) [ 10/Dec/14 ]

There is https://github.com/mongodb/mongo-cxx-driver/wiki/Configuring-the-Legacy-Driver, but it does appear to be out of date. I've triaged this ticket to 1.0.0 and changed the component to 'Documentation'.

Comment by Pieter Willem Jordaan [ 10/Dec/14 ]

Thanks Andrew

I recommend a documentation addition for this, unless I've missed it
somehow.

Comment by Andrew Morrow (Inactive) [ 10/Dec/14 ]

Hi pieterwjordaanpc@gmail.com -

With the code example you have provided, you are correct that you must explicitly invoke client::shutdown() to terminate driver background threads before exiting the process.

You may also, however, request that the driver register an atexit handler to invoke client::shutdown for you, by passing an appropriately configured options structure to client::initialize:

    client::initialize(client::Options().setCallShutdownAtExit());

We don't do this by default, and require you to explicitly terminate the driver because in our testing we found that relying on atexit behavior gave unpredictable results on some platforms.

Additionally, you may be interested in the GlobalInstance class declared in mongo/client/init.h, which uses RAII to schedule the initialization and termination calls for you, without relying on atexit.

If none of those work for you, then you must make an explicit call to 'shutdown'.

Comment by Pieter Jordaan [ 10/Dec/14 ]

Hi Andrew

The following code produces the abort. I assume an explicit shutdown must be called? Is there a way to automatically shutdown on exit?

testabort.cpp:

#include <iostream>
#include <mongo/client/dbclient.h>
 
using namespace std;
using namespace mongo;
 
int main() {
    client::initialize();
    string err;
    ConnectionString cs = ConnectionString::parse("mongodb://localhost:27017/chreosis?replicaSet=myset", err);
    auto connection = cs.connect(err);
// Without the following line I get the abort.
//  client::shutdown();
    return 0;
}

Makefile:

all: main.cpp
	g++ -g -Wall -o testabort -I/usr/local/include main.cpp -L/usr/local/lib -std=c++11 -lboost_system -lboost_thread -lmongoclient
 
clean:
	$(RM) testabort

Comment by Andrew Morrow (Inactive) [ 10/Dec/14 ]

Hi pieterwjordaanpc@gmail.com -

Thank you for the detailed bug report.

If you have a short compilable example which we could try to use to reproduce this crash, that would be ideal. Short of that, if you could provide details for how you are initializing and terminating the library (use of the functions and options defined in mongo/client/init.h and mongo/client/options.h) that too would be helpful.

Looking briefly at the stack traces above, it looks as if static destructors are in progress, but the BackgroundJob (thread) that manages the ReplicaSetMonitorWatcher has not been terminated.

Thanks,
Andrew

Generated at Wed Feb 07 21:59:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.