Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16551

tcmalloc ReleaseToCentralCache account > 14% usage with long insert traffic

    • Type: Icon: Bug Bug
    • Resolution: Won't Fix
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 2.8.0-rc2
    • Component/s: Performance, Storage
    • Labels:
    • Fully Compatible
    • ALL

      during long insert with 64 threads, after about 2500 sec, perf event show tcmalloc's ReleaseToCentralCache is hot, sampled at 14.59% in the following case

      This test is done with wiredtiger, haven;t check mmapv1 yet.

      14.59%    mongod  mongod     [.] tcmalloc::ThreadCache::ReleaseToCentralCache(tcmalloc::ThreadCache::FreeList*, unsigned long, int)
      

      it seems we can adjust max thread cache size to alleviate this situation. The default size is 32M per thread, did test with 64M, which seems improve the throughput

      and here is perf even flame graph, you can check usage interactively. (please download and open in a new window for better view)

        1. perf.data.22ksec.gz
          12.43 MB
        2. perf.data.22ksec.svg
          626 kB
        3. perf.data.4000sec.gz
          8.45 MB
        4. perf.data.4000sec.svg
          722 kB
        5. perf.data.stable-after-bad3.gz
          2.90 MB
        6. perf-kernel-stable-after-bad3.svg
          403 kB
        7. thread_cache_64mb_vs_32mb.png
          thread_cache_64mb_vs_32mb.png
          34 kB

            Assignee:
            geert.bosch@mongodb.com Geert Bosch
            Reporter:
            rui.zhang Rui Zhang (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: