Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-30915

Creating 2dsphere index on collection with documents containing location coordinates consumes high memory

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Won't Do
    • Affects Version/s: 3.2.16, 3.4.7, 3.5.12
    • Fix Version/s: None
    • Component/s: Geo, Indexing
    • Operating System:
      ALL
    • Linked BF Score:
      0

      Description

      On Linux this issue causes the OOM killer to kill a mongod that consumes all memory while creating a geo index. The issue is seen with ps aux output.

      While the createIndex command is in progress, a typical stack is:

      Thread 13 (Thread 0x7f1912370700 (LWP 30347)):
      #0  0x00007f1945d41f71 in S2::IsUnitLength(Vector3<double> const&) ()
      #1  0x00007f1945d649da in S2Loop::AreBoundariesCrossing(S2Loop const*, WedgeProcessor*) const ()
      #2  0x00007f1945d6a92f in S2Loop::Intersects(S2Loop const*) const ()
      #3  0x00007f1945d6aa76 in S2Loop::MayIntersect(S2Cell const&) const ()
      #4  0x00007f1945d7a8b9 in S2Polygon::MayIntersect(S2Cell const&) const ()
      #5  0x00007f1945d8b99b in S2RegionCoverer::NewCandidate(S2Cell const&) ()
      #6  0x00007f1945d8bc5b in S2RegionCoverer::ExpandChildren(S2RegionCoverer::Candidate*, S2Cell const&, int) ()
      #7  0x00007f1945d8bdf6 in S2RegionCoverer::AddCandidate(S2RegionCoverer::Candidate*) ()
      #8  0x00007f1945d8c76c in S2RegionCoverer::GetCoveringInternal(S2Region const&) ()
      #9  0x00007f1945d8d028 in S2RegionCoverer::GetCovering(S2Region const&, std::vector<S2CellId, std::allocator<S2CellId> >*) ()
      #10 0x00007f1945920e53 in (anonymous namespace)::getS2GeoKeys ()
      #11 0x00007f19459217c6 in mongo::ExpressionKeysPrivate::getS2Keys(mongo::BSONObj const&, mongo::BSONObj const&, mongo::S2IndexingParams const&, std::set<mongo::BSONObj, mongo::BSONComparatorInterfaceBase<mongo::BSONObj>::LessThan, std::allocator<mongo::BSONObj> >*, std::vector<std::set<unsigned long, std::less<unsigned long>, std::allocator<unsigned long> >, std::allocator<std::set<unsigned long, std::less<unsigned long>, std::allocator<unsigned long> > > >*) ()
      #12 0x00007f1944dc9f48 in mongo::IndexAccessMethod::getKeys(mongo::BSONObj const&, mongo::IndexAccessMethod::GetKeysMode, std::set<mongo::BSONObj, mongo::BSONComparatorInterfaceBase<mongo::BSONObj>::LessThan, std::allocator<mongo::BSONObj> >*, std::vector<std::set<unsigned long, std::less<unsigned long>, std::allocator<unsigned long> >, std::allocator<std::set<unsigned long, std::less<unsigned long>, std::allocator<unsigned long> > > >*) const ()
      #13 0x00007f1944dcb326 in mongo::IndexAccessMethod::BulkBuilder::insert(mongo::OperationContext*, mongo::BSONObj const&, mongo::RecordId const&, mongo::InsertDeleteOptions const&, long*) ()
      #14 0x00007f1944a1b5e1 in mongo::MultiIndexBlockImpl::insert(mongo::BSONObj const&, mongo::RecordId const&) ()
      #15 0x00007f1944a1baf6 in mongo::MultiIndexBlockImpl::insertAllDocumentsInCollection(std::set<mongo::RecordId, std::less<mongo::RecordId>, std::allocator<mongo::RecordId> >*) ()
      #16 0x00007f19448eea6d in mongo::CmdCreateIndex::errmsgRun(mongo::OperationContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::BSONObj const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, mongo::BSONObjBuilder&) ()
      #17 0x00007f19459b3566 in mongo::ErrmsgCommandDeprecated::run(mongo::OperationContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::BSONObj const&, mongo::BSONObjBuilder&) ()
      #18 0x00007f19459b4ea8 in mongo::BasicCommand::enhancedRun(mongo::OperationContext*, mongo::OpMsgRequest const&, mongo::BSONObjBuilder&) ()
      #19 0x00007f19448c40f4 in mongo::(anonymous namespace)::runCommandImpl ()
      #20 0x00007f19448c5a43 in mongo::(anonymous namespace)::execCommandDatabase ()
      #21 0x00007f19448c71ff in mongo::(anonymous namespace)::runCommands ()
      #22 0x00007f19448c8217 in mongo::ServiceEntryPointMongod::handleRequest(mongo::OperationContext*, mongo::Message const&) ()
      #23 0x00007f19448d0d3e in mongo::ServiceStateMachine::_processMessage(mongo::ServiceStateMachine::ThreadGuard&) ()
      #24 0x00007f19448ceeeb in mongo::ServiceStateMachine::_runNextInGuard(mongo::ServiceStateMachine::ThreadGuard&) ()
      #25 0x00007f19448d067f in mongo::ServiceStateMachine::runNext() ()
      #26 0x00007f19448cbfb1 in std::_Function_handler<void ()(), mongo::ServiceEntryPointImpl::startSession(std::shared_ptr<mongo::transport::Session>)::{lambda()#2}>::_M_invoke(std::_Any_data const&) ()
      #27 0x00007f1945e2ea84 in mongo::(anonymous namespace)::runFunc(void*) ()
      #28 0x00007f194140baa1 in start_thread () from /lib64/libpthread.so.0
      #29 0x00007f1941158bcd in clone () from /lib64/libc.so.6
      

      The thread is continually allocating memory beyond the 500MB suggested limited. On smaller machines this will exhaust all available memory until the OOM killer kicks in. This is reproducible on v3.2 and v3.4, but the memory growth rate is much slower.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: