Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-45137

Increasing memory allocation in Top::record with high rate of collection creates and drops

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 4.2.1
    • Fix Version/s: 4.2.4, 4.3.4
    • Component/s: Catalog
    • Labels:
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Requested:
      v4.2
    • Sprint:
      Execution Team 2020-01-27, Execution Team 2020-02-10
    • Case:

      Description

      Heap profiler shows memory usage increasing over time on the secondary. The profile is characteristic of a buffer that grows by doubling each time.

      Allocated here:

      heapProfile stack155: { 0: "tcmalloc::allocate_full_cpp_throw_oom", 1: "absl::container_internal::raw_hash_set<absl::container_internal::FlatHashMapPolicy<std::__cxx11::basic_string<char, std::char_traits<char>, std::alloc...", 2: "mongo::Top::record", 3: "mongo::AutoStatsTracker::~AutoStatsTracker", 4: "0x5623f753bc58", 5: "0x5623f753caae", 6: "mongo::createCollectionForApplyOps", 7: "0x5623f74c7cc6", 8: "0x5623f74c7ff2", 9: "mongo::repl::applyCommand_inlock", 10: "0x5623f6fa06a4", 11: "mongo::repl::SyncTail::syncApply", 12: "mongo::repl::multiSyncApply", 13: "std::_Function_handler<mongo::Status ", 14: "0x5623f6f9fb5d", 15: "mongo::ThreadPool::_doOneTask", 16: "mongo::ThreadPool::_consumeTasks", 17: "mongo::ThreadPool::_workerThreadBody", 18: "0x5623f87e0a1f", 19: "0x7fa6cc9d26db", 20: "clone" }
      

      Reproduced by a high rate of collection creates and drops using mapReduce directed to the primary:

      function repro() {
       
          db.c.insert({})
       
          nthreads = 10
          threads = []
          for (var t=0; t<nthreads; t++) {
              thread = new ScopedThread(function(t) {
                  count = 1000000
                  for (var i=0; i<count; i++) {
                      db.c.mapReduce(
                          function () {},
                          function () {}, 
                          {out: {merge: "d"}}
                      )
                  }
              }, t)
              threads.push(thread)
              thread.start()
          }
          for (var t = 0; t < nthreads; t++)
              threads[t].join()
       
      }
      

      In this repro the mapReduce commands are done on the primary, but the memory increase only seems to happen on the secondary.

      I don't know whether it is specific to collections created and dropped by mapReduce, and I don't know if it's a 4.2 regression (but I note there appear to have been some considerable code changes in this area).

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              gregory.wlodarek Gregory Wlodarek
              Reporter:
              bruce.lucas Bruce Lucas
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: