Uploaded image for project: 'C Driver'
  1. C Driver
  2. CDRIVER-3522

Stack smashing detected connecting to multiple replica set members with TLS

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 1.18.0, 1.17.3, 1.18.0-alpha
    • Affects Version/s: 1.13.0, 1.14.0
    • Component/s: None
    • Labels:
      None

      This was originally reported in PHPC-1420 but I just now got around to investigating the backtrace. The user originally reported this crash with the following connection string:

      mongodb://demo:PASSWORD@db2.infra.linker.shop:27017,db1.infra.linker.shop:27017,db3.infra.linker.shop:27017/?replicaSet=rps1&ssl=true&authSource=admin

      The ssl and authSource options were originally specified outside of the connection string (via PHP's extra hash argument), but I've merged them in for simplicity.

      The crash itself appeared with the following in debug output from the PHP process:

      *** stack smashing detected ***: php terminated
======= Backtrace: =========
/lib64/libc.so.6(__fortify_fail+0x37)[0x7f2f16295b67]
/lib64/libc.so.6(+0x117b22)[0x7f2f16295b22]
/usr/lib64/php/modules/mongodb.so(mongoc_stream_tls_openssl_new+0x3ea)[0x7f2efe76400a]
/usr/lib64/php/modules/mongodb.so(_mongoc_topology_scanner_node_setup_stream_for_tls+0x2d)[0x7f2efe7698ed]
/usr/lib64/php/modules/mongodb.so(_mongoc_async_cmd_phase_initiate+0x7)[0x7f2efe721e27]
/usr/lib64/php/modules/mongodb.so(mongoc_async_cmd_run+0x33)[0x7f2efe722483]
/usr/lib64/php/modules/mongodb.so(mongoc_async_run+0x36d)[0x7f2efe72180d]
/usr/lib64/php/modules/mongodb.so(mongoc_topology_scanner_work+0x10)[0x7f2efe76a2c0]
/usr/lib64/php/modules/mongodb.so(+0x9139b)[0x7f2efe76439b]
/usr/lib64/php/modules/mongodb.so(_mongoc_topology_do_blocking_scan+0x35)[0x7f2efe764cc5]
/usr/lib64/php/modules/mongodb.so(mongoc_topology_select_server_id+0x337)[0x7f2efe7653b7]
/usr/lib64/php/modules/mongodb.so(mongoc_topology_select+0x11)[0x7f2efe765561]
/usr/lib64/php/modules/mongodb.so(mongoc_client_select_server+0x44)[0x7f2efe729d44]
/usr/lib64/php/modules/mongodb.so(+0xb9692)[0x7f2efe78c692]
/usr/lib64/php/modules/mongodb.so(+0xb9793)[0x7f2efe78c793]

      The equivalent crash in GDB was a bit more verbose, but also includes some function names missing in the previous log:

      Program received signal SIGABRT, Aborted.
0x00007f2f161b42c7 in raise () from /lib64/libc.so.6
(gdb) bt
#0  0x00007f2f161b42c7 in raise () from /lib64/libc.so.6
#1  0x00007f2f161b59b8 in abort () from /lib64/libc.so.6
#2  0x00007f2f161f6e17 in __libc_message () from /lib64/libc.so.6
#3  0x00007f2f16295b67 in __fortify_fail () from /lib64/libc.so.6
#4  0x00007f2f16295b22 in __stack_chk_fail () from /lib64/libc.so.6
#5  0x00007f2efe76400a in mongoc_stream_tls_openssl_new (
    base_stream=base_stream@entry=0x55a498722700, 
    host=0x55a498230c98 "2a01:4f8:173:1828::40", opt=0x55a498231968, 
    client=<optimized out>)
    at /usr/src/debug/php-pecl-mongodb-1.6.0~alpha2/NTS/src/libmongoc/src/libmongoc/src/mongoc/mongoc-stream-tls-openssl.c:777
#6  0x00007f2efe762854 in mongoc_stream_tls_new_with_hostname (
    base_stream=base_stream@entry=0x55a498722700, host=<optimized out>, 
    opt=<optimized out>, client=client@entry=1)
    at /usr/src/debug/php-pecl-mongodb-1.6.0~alpha2/NTS/src/libmongoc/src/libmongoc/src/mongoc/mongoc-stream-tls.c:220
#7  0x00007f2efe7698ed in _mongoc_topology_scanner_node_setup_stream_for_tls (
    node=<optimized out>, stream=0x55a498722700)
    at /usr/src/debug/php-pecl-mongodb-1.6.0~alpha2/NTS/src/libmongoc/src/libmongoc/src/mongoc/mongoc-topology-scanner.c:601
#8  0x00007f2efe721e27 in _mongoc_async_cmd_phase_initiate (
    acmd=0x55a498722020)
    at /usr/src/debug/php-pecl-mongodb-1.6.0~alpha2/NTS/src/libmongoc/src/libmongoc/src/mongoc/mongoc-async-cmd.c:245
#9  0x00007f2efe722483 in mongoc_async_cmd_run (acmd=acmd@entry=0x55a498722020)
    at /usr/src/debug/php-pecl-mongodb-1.6.0~alpha2/NTS/src/libmongoc/src/libmongoc/src/mongoc/mongoc-async-cmd.c:114
#10 0x00007f2efe72180d in mongoc_async_run (async=0x55a49822d520)
    at /usr/src/debug/php-pecl-mongodb-1.6.0~alpha2/NTS/src/libmongoc/src/libmongoc/src/mongoc/mongoc-async.c:94
#11 0x00007f2efe76a2c0 in mongoc_topology_scanner_work (ts=0x55a49822eef0)
    at /usr/src/debug/php-pecl-mongodb-1.6.0~alpha2/NTS/src/libmongoc/src/libmongoc/src/mongoc/mongoc-topology-scanner.c:1007
#12 0x00007f2efe76439b in mongoc_topology_scan_once (
    topology=topology@entry=0x55a49822de90, 
    obey_cooldown=obey_cooldown@entry=true)
    at /usr/src/debug/php-pecl-mongodb-1.6.0~alpha2/NTS/src/libmongoc/src/libmongoc/src/mongoc/mongoc-topology.c:471
#13 0x00007f2efe764cc5 in _mongoc_topology_do_blocking_scan (
    topology=topology@entry=0x55a49822de90, error=error@entry=0x7fff34f386a0)
    at /usr/src/debug/php-pecl-mongodb-1.6.0~alpha2/NTS/src/libmongoc/src/libmongoc/src/mongoc/mongoc-topology.c:501
#14 0x00007f2efe7653b7 in mongoc_topology_select_server_id (
    topology=topology@entry=0x55a49822de90, 
    optype=optype@entry=MONGOC_SS_READ, 
    read_prefs=read_prefs@entry=0x55a49822d710, 
    error=error@entry=0x7fff34f38950)
    at /usr/src/debug/php-pecl-mongodb-1.6.0~alpha2/NTS/src/libmongoc/src/libmongoc/src/mongoc/mongoc-topology.c:734
#15 0x00007f2efe765561 in mongoc_topology_select (topology=0x55a49822de90, 
    optype=optype@entry=MONGOC_SS_READ, 
    read_prefs=read_prefs@entry=0x55a49822d710, 
    error=error@entry=0x7fff34f38950)
    at /usr/src/debug/php-pecl-mongodb-1.6.0~alpha2/NTS/src/libmongoc/src/libmongoc/src/mongoc/mongoc-topology.c:616
#16 0x00007f2efe729d44 in mongoc_client_select_server (
    client=client@entry=0x55a4982318e0, for_writes=for_writes@entry=false, 
    prefs=0x55a49822d710, error=error@entry=0x7fff34f38950)
    at /usr/src/debug/php-pecl-mongodb-1.6.0~alpha2/NTS/src/libmongoc/src/libmongoc/src/mongoc/mongoc-client.c:2620
#17 0x00007f2efe78c692 in php_phongo_manager_select_server (
    for_writes=for_writes@entry=false, zreadPreference=<optimized out>, 
    client=0x55a4982318e0, server_id=server_id@entry=0x7fff34f38b9c)
    at /usr/src/debug/php-pecl-mongodb-1.6.0~alpha2/NTS/src/MongoDB/Manager.c:271
#18 0x00007f2efe78c793 in zim_Manager_selectServer (
    execute_data=<optimized out>, return_value=0x7f2f14e15210)
    at /usr/src/debug/php-pecl-mongodb-1.6.0~alpha2/NTS/src/MongoDB/Manager.c:631

      Researching __stack_chk_fail suggests that it's merely indicative of when the stack smash was detected, but not where it originated. This Stack Overflow thread suggests using AddressSanitizer to investigate further. Without being able to identify the smash with ASan, I believe anything in the stack trace could be suspect.

      The crash itself was reported against PHPC 1.5.5 and 1.6.0alpha2, so I've marked the affected version as libmongoc 1.13.0 and 1.14.0, respectively. In the meantime, I've also asked the user to attempt reproduction using the latest version of the driver (1.7.1), which uses libmongoc 1.16.1.

            Assignee:
            roberto.sanchez@mongodb.com Roberto Sanchez
            Reporter:
            jmikola@mongodb.com Jeremy Mikola
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: