Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16654

ParallelConnectionMetadata::cleanup can cause the cursor to access deleted connection

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 2.4.12, 2.8.0-rc3
    • Component/s: Sharding
    • Labels:
      None
    • Sharding
    • ALL
    • Sharding C (11/20/15)

      ParallelConnectionMetadata::cleanup calls ShardConnection::done, which can potentially delete the underlying connection (https://github.com/mongodb/mongo/blob/r2.8.0-rc3/src/mongo/client/connpool.cpp#L66-73). This is also the same connection pointer stored in the cursor and when it's destructor gets called, it can try to access the pointer (https://github.com/mongodb/mongo/blob/r2.8.0-rc3/src/mongo/client/dbclientcursor.cpp#L364-372).

      Sample stacktrace from special local 2.4.12 build with special fail points to make it fail easier:

       m30999| /home/ren/mongo-copy/mongos(_ZN5mongo17printStackAndExitEi+0x65)[0x965f65]
       m30999| /lib/x86_64-linux-gnu/libc.so.6(+0x370b0)[0x7f3885d7d0b0]
       m30999| /lib/x86_64-linux-gnu/libc.so.6(+0x150fe9)[0x7f3885e96fe9]
       m30999| /home/ren/mongo-copy/mongos(_ZNK5mongo18DBClientReplicaSet11_getMonitorEv+0xa9)[0x6b8269]
       m30999| /home/ren/mongo-copy/mongos(_ZN5mongo18DBClientReplicaSet11checkMasterEv+0x24)[0x6b8314]
       m30999| /home/ren/mongo-copy/mongos(_ZN5mongo18DBClientReplicaSet3sayERNS_7MessageEbPSs+0x7e)[0x6bbc6e]
       m30999| /home/ren/mongo-copy/mongos(_ZN5mongo14DBClientCursorD1Ev+0x578)[0x6c6238]
       m30999| /home/ren/mongo-copy/mongos(_ZN5mongo14DBClientCursorD0Ev+0x9)[0x6c65e9]
       m30999| /home/ren/mongo-copy/mongos(_ZN5boost6detail12shared_countD2Ev+0x39)[0x6736c9]
       m30999| /home/ren/mongo-copy/mongos(_ZN5boost6detail17sp_counted_impl_pIN5mongo23ParallelConnectionStateEE7disposeEv+0x2a)[0x6f6efa]
       m30999| /home/ren/mongo-copy/mongos(_ZN5boost6detail12shared_countD2Ev+0x39)[0x6736c9]
       m30999| /home/ren/mongo-copy/mongos(_ZN5mongo26ParallelConnectionMetadata7cleanupEb+0x172)[0x6e47a2]
       m30999| /home/ren/mongo-copy/mongos(_ZNSt8_Rb_treeIN5mongo5ShardESt4pairIKS1_NS0_26ParallelConnectionMetadataEESt10_Select1stIS5_ESt4lessIS1_ESaIS5_EE8_M_eraseEPSt13_Rb_tree_nodeIS5_E+0x4b)[0x6f84bb]
       m30999| /home/ren/mongo-copy/mongos(_ZN5mongo27ParallelSortClusteredCursorD1Ev+0xcd)[0x6e585d]
       m30999| /home/ren/mongo-copy/mongos(_ZN5mongo27ParallelSortClusteredCursorD0Ev+0x9)[0x6e59e9]
       m30999| /home/ren/mongo-copy/mongos(_ZN5mongo13ShardStrategy7queryOpERNS_7RequestE+0x11cd)[0x8c434d]
       m30999| /home/ren/mongo-copy/mongos(_ZN5mongo7Request7processEi+0x18f)[0x8aa74f]
       m30999| /home/ren/mongo-copy/mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x60)[0x679090]
       m30999| /home/ren/mongo-copy/mongos(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x471)[0x951501]
       m30999| /lib/x86_64-linux-gnu/libpthread.so.0(+0x7f8e)[0x7f3886b44f8e]
      

            Assignee:
            backlog-server-sharding [DO NOT USE] Backlog - Sharding Team
            Reporter:
            randolph@mongodb.com Randolph Tan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: