Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-25075

Building 2dsphere index uses excessive memory

    XMLWordPrintable

Details

    • Fully Compatible
    • ALL
    • Repl 18 (08/05/16)

    Description

      Create a collection with 24 million documents {loc: [0, 0]}. Then create an index with db.c.createIndex({loc:"2dsphere"}).

      • index create runs from A to B
      • 1825 MB allocated outside the cache ("allocated minus wt cache")
      • heap profile shows the following stack accounts for most of the allocations:

        heapProfile stack39: { 0: "tc_malloc", 1: "mongo::mongoMalloc", 2: "mongo::S2CellIdToIndexKey", 3: "0xc79010", 4: "mongo::ExpressionKeysPrivate::getS2Keys", 5: "mongo::IndexAccessMethod::BulkBuilder::insert", 6: "mongo::MultiIndexBlock::insert", 7: "mongo::MultiIndexBlock::insertAllDocumentsInCollection", 8: "mongo::CmdCreateIndex::run", 9: "mongo::Command::run", 10: "mongo::Command::execCommand", 11: "mongo::runCommands", 12: "mongo::assembleResponse", 13: "mongo::MyMessageHandler::process", 14: "mongo::PortMessageServer::handleIncomingMsg", 15: "0x7fd30dc996aa", 16: "clone" }
        

        These are keys being accumulated for sorting. The accumulated size of the keys is growing to 1348 MiB before external sort is used whereas it should be limited to 100 MiB.

      Attachments

        Issue Links

          Activity

            People

              siyuan.zhou@mongodb.com Siyuan Zhou
              bruce.lucas@mongodb.com Bruce Lucas
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: