Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-19991

Use-after-free in DBClientCursor::kill when performing sharded mapReduce

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 3.1.9
    • Affects Version/s: 3.1.4
    • Component/s: MapReduce, Sharding
    • Labels:
    • Fully Compatible
    • ALL
    • Quint 9 09/18/15, QuInt A (10/12/15)

      This bug appears to be a regression in 3.1.x. I can't seem to trigger the use-after-free in 3.0.5.

      ==3108== ERROR: AddressSanitizer: heap-use-after-free on address 0x602400821e40 at pc 0x1b6d93e bp 0x7f6ac9155f90 sp 0x7f6ac9155f88
      READ of size 8 at 0x602400821e40 thread T114
      ==3108== AddressSanitizer: while reporting a bug found another one.Ignoring.
          #0 0x1b6d93d in mongo::DBClientCursor::kill() /home/s/code/mongo/mongo/src/mongo/client/dbclientcursor.cpp:508
          #1 0x1b6d728 in mongo::DBClientCursor::~DBClientCursor() /home/s/code/mongo/mongo/src/mongo/client/dbclientcursor.cpp:504
          #2 0x1b6d7d7 in mongo::DBClientCursor::~DBClientCursor() /home/s/code/mongo/mongo/src/mongo/client/dbclientcursor.cpp:505
          #3 0x1b53356 in std::default_delete<mongo::DBClientCursor>::operator()(mongo::DBClientCursor*) const /usr/include/c++/4.8/bits/unique_ptr.h:67
          #4 0x1b4ff2d in std::unique_ptr<mongo::DBClientCursor, std::default_delete<mongo::DBClientCursor> >::~unique_ptr() /usr/include/c++/4.8/bits/unique_ptr.h:184
          #5 0x1b8ce39 in mongo::DBClientCursorHolder::~DBClientCursorHolder() /home/s/code/mongo/mongo/src/mongo/client/parallel.h:280
          #6 0x1b89f23 in mongo::ParallelSortClusteredCursor::~ParallelSortClusteredCursor() /home/s/code/mongo/mongo/src/mongo/client/parallel.cpp:1434
          #7 0x1d54c73 in mongo::mr::MapReduceFinishCommand::run(mongo::OperationContext*, std::string const&, mongo::BSONObj&, int, std::string&, mongo::BSONObjBuilder&) /home/s/code/mongo/mongo/src/mongo/db/commands/mr.cpp:1672
          #8 0x1defc29 in mongo::Command::run(mongo::OperationContext*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*) /home/s/code/mongo/mongo/src/mongo/db/dbcommands.cpp:1346
          #9 0x1deed49 in mongo::Command::execCommand(mongo::OperationContext*, mongo::Command*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*) /home/s/code/mongo/mongo/src/mongo/db/dbcommands.cpp:1266
          #10 0x1cf6b33 in mongo::runCommands(mongo::OperationContext*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*) /home/s/code/mongo/mongo/src/mongo/db/commands.cpp:495
          #11 0x1fce494 in mongo::(anonymous namespace)::receivedRpc(mongo::OperationContext*, mongo::Client&, mongo::DbResponse&, mongo::Message&) /home/s/code/mongo/mongo/src/mongo/db/instance.cpp:290
          #12 0x1fcfaba in mongo::assembleResponse(mongo::OperationContext*, mongo::Message&, mongo::DbResponse&, mongo::HostAndPort const&) /home/s/code/mongo/mongo/src/mongo/db/instance.cpp:508
          #13 0x1a9c14e in mongo::MyMessageHandler::process(mongo::Message&, mongo::AbstractMessagingPort*) /home/s/code/mongo/mongo/src/mongo/db/db.cpp:165
          #14 0x291531d in mongo::PortMessageServer::handleIncomingMsg(void*) /home/s/code/mongo/mongo/src/mongo/util/net/message_server_port.cpp:229
          #15 0x7f6b01db1b97 (/usr/lib/x86_64-linux-gnu/libasan.so.0+0x18b97)
          #16 0x7f6b00f57181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312
          #17 0x7f6b00c8447c in clone /build/buildd/eglibc-2.19/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:111
      0x602400821e40 is located 0 bytes inside of 256-byte region [0x602400821e40,0x602400821f40)
      freed by thread T114 here:
          #0 0x7f6b01daa9da in operator delete(void*) (/usr/lib/x86_64-linux-gnu/libasan.so.0+0x119da)
          #1 0x1b4cc75 in mongo::DBClientConnection::~DBClientConnection() /home/s/code/mongo/mongo/src/mongo/client/dbclientinterface.h:1118
          #2 0x1b306d2 in mongo::ScopedDbConnection::kill() /home/s/code/mongo/mongo/src/mongo/client/connpool.h:367
          #3 0x1b2f876 in mongo::ScopedDbConnection::~ScopedDbConnection() /home/s/code/mongo/mongo/src/mongo/client/connpool.cpp:511
          #4 0x1b6a81a in mongo::DBClientCursor::requestMore() /home/s/code/mongo/mongo/src/mongo/client/dbclientcursor.cpp:221
          #5 0x1b6bf69 in mongo::DBClientCursor::more() /home/s/code/mongo/mongo/src/mongo/client/dbclientcursor.cpp:341
          #6 0x1b8a63b in mongo::ParallelSortClusteredCursor::next() /home/s/code/mongo/mongo/src/mongo/client/parallel.cpp:1482
          #7 0x1d54359 in mongo::mr::MapReduceFinishCommand::run(mongo::OperationContext*, std::string const&, mongo::BSONObj&, int, std::string&, mongo::BSONObjBuilder&) /home/s/code/mongo/mongo/src/mongo/db/commands/mr.cpp:1679
          #8 0x1defc29 in mongo::Command::run(mongo::OperationContext*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*) /home/s/code/mongo/mongo/src/mongo/db/dbcommands.cpp:1346
          #9 0x1deed49 in mongo::Command::execCommand(mongo::OperationContext*, mongo::Command*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*) /home/s/code/mongo/mongo/src/mongo/db/dbcommands.cpp:1266
          #10 0x1cf6b33 in mongo::runCommands(mongo::OperationContext*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*) /home/s/code/mongo/mongo/src/mongo/db/commands.cpp
          #11 0x1fce494 in mongo::(anonymous namespace)::receivedRpc(mongo::OperationContext*, mongo::Client&, mongo::DbResponse&, mongo::Message&) /home/s/code/mongo/mongo/src/mongo/db/instance.cpp:290
          #12 0x1fcfaba in mongo::assembleResponse(mongo::OperationContext*, mongo::Message&, mongo::DbResponse&, mongo::HostAndPort const&) /home/s/code/mongo/mongo/src/mongo/db/instance.cpp:508
          #13 0x1a9c14e in mongo::MyMessageHandler::process(mongo::Message&, mongo::AbstractMessagingPort*) /home/s/code/mongo/mongo/src/mongo/db/db.cpp:165
          #14 0x291531d in mongo::PortMessageServer::handleIncomingMsg(void*) /home/s/code/mongo/mongo/src/mongo/util/net/message_server_port.cpp:229
          #15 0x7f6b01db1b97 (/usr/lib/x86_64-linux-gnu/libasan.so.0+0x18b97)
      previously allocated by thread T114 here:
          #0 0x7f6b01daa81a in operator new(unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.0+0x1181a)
          #1 0x1b2a380 in boost::detail::up_if_not_array<mongo::DBClientConnection>::type boost::make_unique<mongo::DBClientConnection, bool>(bool&&) /home/s/code/mongo/mongo/src/third_party/boost-1.56.0/boost/smart_ptr/make_unique_object.hpp:28
          #2 0x1b29a22 in mongo::ConnectionString::connect(std::string&, double) const /home/s/code/mongo/mongo/src/mongo/client/connection_string_connect.cpp:52
          #3 0x1b2ca01 in mongo::DBConnectionPool::get(std::string const&, double) /home/s/code/mongo/mongo/src/mongo/client/connpool.cpp
          #4 0x1b2f0f8 in mongo::ScopedDbConnection::ScopedDbConnection(std::string const&, double) /home/s/code/mongo/mongo/src/mongo/client/connpool.cpp:468
          #5 0x1b6a641 in mongo::DBClientCursor::requestMore() /home/s/code/mongo/mongo/src/mongo/client/dbclientcursor.cpp:215
          #6 0x1b6bf69 in mongo::DBClientCursor::more() /home/s/code/mongo/mongo/src/mongo/client/dbclientcursor.cpp:341
          #7 0x1b8a63b in mongo::ParallelSortClusteredCursor::next() /home/s/code/mongo/mongo/src/mongo/client/parallel.cpp:1482
          #8 0x1d54359 in mongo::mr::MapReduceFinishCommand::run(mongo::OperationContext*, std::string const&, mongo::BSONObj&, int, std::string&, mongo::BSONObjBuilder&) /home/s/code/mongo/mongo/src/mongo/db/commands/mr.cpp:1679
          #9 0x1defc29 in mongo::Command::run(mongo::OperationContext*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*) /home/s/code/mongo/mongo/src/mongo/db/dbcommands.cpp:1346
          #10 0x1deed49 in mongo::Command::execCommand(mongo::OperationContext*, mongo::Command*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*) /home/s/code/mongo/mongo/src/mongo/db/dbcommands.cpp:1266
          #11 0x1cf6b33 in mongo::runCommands(mongo::OperationContext*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*) /home/s/code/mongo/mongo/src/mongo/db/commands.cpp:495
          #12 0x1fce494 in mongo::(anonymous namespace)::receivedRpc(mongo::OperationContext*, mongo::Client&, mongo::DbResponse&, mongo::Message&) /home/s/code/mongo/mongo/src/mongo/db/instance.cpp:290
          #13 0x1fcfaba in mongo::assembleResponse(mongo::OperationContext*, mongo::Message&, mongo::DbResponse&, mongo::HostAndPort const&) /home/s/code/mongo/mongo/src/mongo/db/instance.cpp:508
          #14 0x1a9c14e in mongo::MyMessageHandler::process(mongo::Message&, mongo::AbstractMessagingPort*) /home/s/code/mongo/mongo/src/mongo/db/db.cpp:165
          #15 0x291531d in mongo::PortMessageServer::handleIncomingMsg(void*) /home/s/code/mongo/mongo/src/mongo/util/net/message_server_port.cpp:229
          #16 0x7f6b01db1b97 (/usr/lib/x86_64-linux-gnu/libasan.so.0+0x18b97)
      Thread T114 created by T0 here:
          #0 0x7f6b01da3b5b in __interceptor_pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.0+0xab5b)
          #1 0x2914a0d in mongo::PortMessageServer::accepted(std::shared_ptr<mongo::Socket>, long long) /home/s/code/mongo/mongo/src/mongo/util/net/message_server_port.cpp:148
          #2 0x290adcc in mongo::Listener::initAndListen() /home/s/code/mongo/mongo/src/mongo/util/net/listen.cpp:351
          #3 0x2914d2d in mongo::PortMessageServer::run() /home/s/code/mongo/mongo/src/mongo/util/net/message_server_port.cpp:176
          #4 0x1a93bcd in mongo::_initAndListen(int) /home/s/code/mongo/mongo/src/mongo/db/db.cpp:588
          #5 0x1a93fee in mongo::initAndListen(int) /home/s/code/mongo/mongo/src/mongo/db/db.cpp:593
          #6 0x1a9599e in mongoDbMain(int, char**, char**) /home/s/code/mongo/mongo/src/mongo/db/db.cpp:833
          #7 0x1a94398 in main /home/s/code/mongo/mongo/src/mongo/db/db.cpp:638
          #8 0x7f6b00babec4 in __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:287
      SUMMARY: AddressSanitizer: heap-use-after-free /home/s/code/mongo/mongo/src/mongo/client/dbclientcursor.cpp:508 mongo::DBClientCursor::kill()
      Shadow bytes around the buggy address:
        0x0c05000fc370: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
        0x0c05000fc380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
        0x0c05000fc390: 00 00 00 00 00 00 00 00 fa fa fa fa fa fa fa fa
        0x0c05000fc3a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
        0x0c05000fc3b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      =>0x0c05000fc3c0: fa fa fa fa fa fa fa fa[fd]fd fd fd fd fd fd fd
        0x0c05000fc3d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x0c05000fc3e0: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa
        0x0c05000fc3f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x0c05000fc400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x0c05000fc410: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      Shadow byte legend (one shadow byte represents 8 application bytes):
        Addressable:           00
        Partially addressable: 01 02 03 04 05 06 07
        Heap left redzone:     fa
        Heap righ redzone:     fb
        Freed Heap region:     fd
        Stack left redzone:    f1
        Stack mid redzone:     f2
        Stack right redzone:   f3
        Stack partial redzone: f4
        Stack after return:    f5
        Stack use after scope: f8
        Global redzone:        f9
        Global init order:     f6
        Poisoned by user:      f7
        ASan internal:         fe
      ==3108== ABORTING
      

      Version: ea25f4c281237d4b43f86a3ca8c9d1fddeaeb8fc

            Assignee:
            charlie.swanson@mongodb.com Charlie Swanson
            Reporter:
            kamran.khan Kamran K.
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: