Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16551

tcmalloc ReleaseToCentralCache account > 14% usage with long insert traffic

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Won't Fix
    • 2.8.0-rc2
    • None
    • Performance, Storage
    • Fully Compatible
    • ALL

    Description

      during long insert with 64 threads, after about 2500 sec, perf event show tcmalloc's ReleaseToCentralCache is hot, sampled at 14.59% in the following case

      This test is done with wiredtiger, haven;t check mmapv1 yet.

      14.59%    mongod  mongod     [.] tcmalloc::ThreadCache::ReleaseToCentralCache(tcmalloc::ThreadCache::FreeList*, unsigned long, int)

      it seems we can adjust max thread cache size to alleviate this situation. The default size is 32M per thread, did test with 64M, which seems improve the throughput

      and here is perf even flame graph, you can check usage interactively. (please download and open in a new window for better view)

      Attachments

        1. perf.data.22ksec.gz
          12.43 MB
        2. perf.data.22ksec.svg
          626 kB
        3. perf.data.4000sec.gz
          8.45 MB
        4. perf.data.4000sec.svg
          722 kB
        5. perf.data.stable-after-bad3.gz
          2.90 MB
        6. perf-kernel-stable-after-bad3.svg
          403 kB
        7. thread_cache_64mb_vs_32mb.png
          thread_cache_64mb_vs_32mb.png
          34 kB

        Issue Links

          Activity

            People

              geert.bosch@mongodb.com Geert Bosch
              rui.zhang Rui Zhang (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: