Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-25075

Building 2dsphere index uses excessive memory

    • Fully Compatible
    • ALL
    • Repl 18 (08/05/16)

      Create a collection with 24 million documents {loc: [0, 0]}. Then create an index with db.c.createIndex({loc:"2dsphere"}).

      • index create runs from A to B
      • 1825 MB allocated outside the cache ("allocated minus wt cache")
      • heap profile shows the following stack accounts for most of the allocations:
        heapProfile stack39: { 0: "tc_malloc", 1: "mongo::mongoMalloc", 2: "mongo::S2CellIdToIndexKey", 3: "0xc79010", 4: "mongo::ExpressionKeysPrivate::getS2Keys", 5: "mongo::IndexAccessMethod::BulkBuilder::insert", 6: "mongo::MultiIndexBlock::insert", 7: "mongo::MultiIndexBlock::insertAllDocumentsInCollection", 8: "mongo::CmdCreateIndex::run", 9: "mongo::Command::run", 10: "mongo::Command::execCommand", 11: "mongo::runCommands", 12: "mongo::assembleResponse", 13: "mongo::MyMessageHandler::process", 14: "mongo::PortMessageServer::handleIncomingMsg", 15: "0x7fd30dc996aa", 16: "clone" }
        

        These are keys being accumulated for sorting. The accumulated size of the keys is growing to 1348 MiB before external sort is used whereas it should be limited to 100 MiB.

        1. 2dsphere.png
          97 kB
          Bruce Lucas

            Assignee:
            siyuan.zhou@mongodb.com Siyuan Zhou
            Reporter:
            bruce.lucas@mongodb.com Bruce Lucas (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: