Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-25075

Building 2dsphere index uses excessive memory

    Details

    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Completed:
    • Sprint:
      Repl 18 (08/05/16)

      Description

      Create a collection with 24 million documents {loc: [0, 0]}. Then create an index with db.c.createIndex({loc:"2dsphere"}).

      • index create runs from A to B
      • 1825 MB allocated outside the cache ("allocated minus wt cache")
      • heap profile shows the following stack accounts for most of the allocations:

        heapProfile stack39: { 0: "tc_malloc", 1: "mongo::mongoMalloc", 2: "mongo::S2CellIdToIndexKey", 3: "0xc79010", 4: "mongo::ExpressionKeysPrivate::getS2Keys", 5: "mongo::IndexAccessMethod::BulkBuilder::insert", 6: "mongo::MultiIndexBlock::insert", 7: "mongo::MultiIndexBlock::insertAllDocumentsInCollection", 8: "mongo::CmdCreateIndex::run", 9: "mongo::Command::run", 10: "mongo::Command::execCommand", 11: "mongo::runCommands", 12: "mongo::assembleResponse", 13: "mongo::MyMessageHandler::process", 14: "mongo::PortMessageServer::handleIncomingMsg", 15: "0x7fd30dc996aa", 16: "clone" }
        

        These are keys being accumulated for sorting. The accumulated size of the keys is growing to 1348 MiB before external sort is used whereas it should be limited to 100 MiB.

        Attachments

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: