[connXXXX] out of memory crashing the server

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Incomplete
    • Priority: Major - P3
    • None
    • Affects Version/s: 4.2.6, 4.2.13
    • Component/s: None
    • None
    • ALL
    • Hide

      So far, I was not able to reproduce the issue at will.

      I will see if I can get a core dump. FTDC data does not report anything unusual during the moment of the crash.

      Show
      So far, I was not able to reproduce the issue at will. I will see if I can get a core dump. FTDC data does not report anything unusual during the moment of the crash.
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Hi,

      I'm facing an issue where using MongoDB 4.2.12 (I also observed in 4.2.6) crashing with the following backtrace:

      mongod(mongo::printStackTrace(std::basic_ostream<char, std::char_traits<char> >&)+0x41) [0x55e8198e4f61]
      mongod(mongo::reportOutOfMemoryErrorAndExit()+0x87) [0x55e8198e4807]
      mongod(+0x2BC6631) [0x55e8199f0631]
      mongod(tcmalloc::allocate_full_cpp_throw_oom(unsigned long)+0xC9) [0x55e819b79129]
      mongod(std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&)+0x61) [0x55e817c913d1]
      mongod(std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Rep::_M_clone(std::allocator<char> const&, unsigned long)+0x1C) [0x55e817c9160c]
      mongod(mongo::rpc::getImpersonatedUserMetadata(mongo::OperationContext*)+0x22B) [0x55e81961a13b]
      mongod(mongo::CurOp::reportCurrentOpForClient(mongo::OperationContext*, mongo::Client*, bool, bool, mongo::BSONObjBuilder*)+0x3EA) [0x55e81923e47a]
      mongod(mongo::MongoInterfaceStandalone::_reportCurrentOpForClient(mongo::OperationContext*, mongo::Client*, mongo::MongoProcessInterface::CurrentOpTruncateMode, mongo::MongoProcessInterface::CurrentOpBacktraceMode) const+0x70) [0x55e8180dff80]
      mongod(mongo::MongoProcessCommon::getCurrentOps(boost::intrusive_ptr<mongo::ExpressionContext> const&, mongo::MongoProcessInterface::CurrentOpConnectionsMode, mongo::MongoProcessInterface::CurrentOpSessionsMode, mongo::MongoProcessInterface::CurrentOpUserMode, mongo::MongoProcessInterface::CurrentOpTruncateMode, mongo::MongoProcessInterface::CurrentOpCursorMode, mongo::MongoProcessInterface::CurrentOpBacktraceMode) const+0xFB) [0x55e8182f101b]
      mongod(mongo::DocumentSourceCurrentOp::getNext()+0x82A) [0x55e81917cefa]
      mongod(mongo::DocumentSourceMatch::getNext()+0x56) [0x55e8191ba1a6]
      mongod(mongo::DocumentSourceSingleDocumentTransformation::getNext()+0x38) [0x55e8191e7c98]
      mongod(mongo::Pipeline::getNext()+0x3D) [0x55e8191f983d]
      mongod(mongo::PipelineProxyStage::getNextBson()+0x35) [0x55e81875ab35]
      mongod(mongo::PipelineProxyStage::doWork(unsigned long*)+0x46) [0x55e81875a9a6]
      mongod(mongo::PlanStage::work(unsigned long*)+0x64) [0x55e81875b0e4]
      mongod(mongo::PlanExecutorImpl::_getNextImpl(mongo::Snapshotted<mongo::BSONObj>*, mongo::RecordId*)+0x22D) [0x55e8187a452d]
      mongod(mongo::PlanExecutorImpl::getNext(mongo::BSONObj*, mongo::RecordId*)+0x4D) [0x55e8187a4c8d]
      mongod(+0x16290E9) [0x55e8184530e9]
      mongod(mongo::runAggregate(mongo::OperationContext*, mongo::NamespaceString const&, mongo::AggregationRequest const&, mongo::BSONObj const&, std::vector<mongo::Privilege, std::allocator<mongo::Privilege> > const&, mongo::rpc::ReplyBuilderInterface*)+0x2327) [0x55e8184574e7]
      mongod(+0x1620412) [0x55e81844a412]
      mongod(+0x12D5887) [0x55e8180ff887]
      mongod(+0x12D7BEB) [0x55e818101beb]
      mongod(mongo::ServiceEntryPointCommon::handleRequest(mongo::OperationContext*, mongo::Message const&, mongo::ServiceEntryPointCommon::Hooks const&)+0x4E7) [0x55e818102a07]
      mongod(mongo::ServiceEntryPointMongod::handleRequest(mongo::OperationContext*, mongo::Message const&)+0x3C) [0x55e8180efcdc]
      mongod(mongo::ServiceStateMachine::_processMessage(mongo::ServiceStateMachine::ThreadGuard)+0xEF) [0x55e8180fc8bf]
      mongod(mongo::ServiceStateMachine::_runNextInGuard(mongo::ServiceStateMachine::ThreadGuard)+0xF6) [0x55e8180f9b66]
      mongod(+0x12D193B) [0x55e8180fb93b]
      mongod(mongo::transport::ServiceExecutorSynchronous::schedule(std::function<void ()>, mongo::transport::ServiceExecutor::ScheduleFlags, mongo::transport::ServiceExecutorTaskName)+0x182) [0x55e818fc9642]
      mongod(mongo::ServiceStateMachine::_scheduleNextWithGuard(mongo::ServiceStateMachine::ThreadGuard, mongo::transport::ServiceExecutor::ScheduleFlags, mongo::transport::ServiceExecutorTaskName, mongo::ServiceStateMachine::Ownership)+0x119) [0x55e8180f8159]
      mongod(mongo::ServiceStateMachine::_sourceCallback(mongo::Status)+0x4C4) [0x55e8180fa0a4]
      mongod(mongo::ServiceStateMachine::_sourceMessage(mongo::ServiceStateMachine::ThreadGuard)+0x2A6) [0x55e8180fac86]
      mongod(mongo::ServiceStateMachine::_runNextInGuard(mongo::ServiceStateMachine::ThreadGuard)+0xC2) [0x55e8180f9b32]
      mongod(+0x12D193B) [0x55e8180fb93b]
      mongod(+0x219FAA7) [0x55e818fc9aa7]
      mongod(+0x2826DD5) [0x55e819650dd5]
      mongod(+0x2826E34) [0x55e819650e34]
      libpthread.so.0(+0x7EA5) [0x7f02c0b96ea5]
      libc.so.6(clone+0x6D) [0x7f02c08bf8dd]
      

      Sometimes it also crashes with SIGSEGV.

      The query that was running:

      db.adminCommand( {"aggregate": 1, pipeline:[{"$currentOp": { "allUsers": true, "idleConnections": false }},
      {"$match": {"ns": {"$nin":["", "local.$cmd", "local.oplog.rs", "admin.$cmd.aggregate", "admin.$cmd", "local.replset.minvalid"]},
      "op": {"$nin":["","none"]}}},
      {"$addFields": {"currentOpTime":
      {"$dateFromString": {"dateString": "$currentOpTime"}}}}], "explain":true })
      

      Downgrading to 4.0 or upgrading solves the issue.

      Sometimes the connection and Mongo crashes with SIGSEV:

      2021-07-22T12:29:54.119+0900 F  -        [conn460933] Invalid access at address: 0
      2021-07-22T12:29:54.237+0900 F  -        [conn460933] Got signal: 11 (Segmentation fault).
      

            Assignee:
            Dmitry Agranat
            Reporter:
            Vinicius Grippa
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: