Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-45137

Increasing memory allocation in Top::record with high rate of collection creates and drops

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • 4.2.1
    • 4.2.4, 4.3.4
    • Catalog
    • Fully Compatible
    • ALL
    • v4.2
    • Execution Team 2020-01-27, Execution Team 2020-02-10

    Description

      Heap profiler shows memory usage increasing over time on the secondary. The profile is characteristic of a buffer that grows by doubling each time.

      Allocated here:

      heapProfile stack155: { 0: "tcmalloc::allocate_full_cpp_throw_oom", 1: "absl::container_internal::raw_hash_set<absl::container_internal::FlatHashMapPolicy<std::__cxx11::basic_string<char, std::char_traits<char>, std::alloc...", 2: "mongo::Top::record", 3: "mongo::AutoStatsTracker::~AutoStatsTracker", 4: "0x5623f753bc58", 5: "0x5623f753caae", 6: "mongo::createCollectionForApplyOps", 7: "0x5623f74c7cc6", 8: "0x5623f74c7ff2", 9: "mongo::repl::applyCommand_inlock", 10: "0x5623f6fa06a4", 11: "mongo::repl::SyncTail::syncApply", 12: "mongo::repl::multiSyncApply", 13: "std::_Function_handler<mongo::Status ", 14: "0x5623f6f9fb5d", 15: "mongo::ThreadPool::_doOneTask", 16: "mongo::ThreadPool::_consumeTasks", 17: "mongo::ThreadPool::_workerThreadBody", 18: "0x5623f87e0a1f", 19: "0x7fa6cc9d26db", 20: "clone" }
      

      Reproduced by a high rate of collection creates and drops using mapReduce directed to the primary:

      function repro() {
       
          db.c.insert({})
       
          nthreads = 10
          threads = []
          for (var t=0; t<nthreads; t++) {
              thread = new ScopedThread(function(t) {
                  count = 1000000
                  for (var i=0; i<count; i++) {
                      db.c.mapReduce(
                          function () {},
                          function () {}, 
                          {out: {merge: "d"}}
                      )
                  }
              }, t)
              threads.push(thread)
              thread.start()
          }
          for (var t = 0; t < nthreads; t++)
              threads[t].join()
       
      }
      

      In this repro the mapReduce commands are done on the primary, but the memory increase only seems to happen on the secondary.

      I don't know whether it is specific to collections created and dropped by mapReduce, and I don't know if it's a 4.2 regression (but I note there appear to have been some considerable code changes in this area).

      Attachments

        Issue Links

          Activity

            People

              gregory.wlodarek@mongodb.com Gregory Wlodarek
              bruce.lucas@mongodb.com Bruce Lucas
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: