Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16551

tcmalloc ReleaseToCentralCache account > 14% usage with long insert traffic

    • Type: Icon: Bug Bug
    • Resolution: Won't Fix
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 2.8.0-rc2
    • Component/s: Performance, Storage
    • Labels:
    • Fully Compatible
    • ALL

      during long insert with 64 threads, after about 2500 sec, perf event show tcmalloc's ReleaseToCentralCache is hot, sampled at 14.59% in the following case

      This test is done with wiredtiger, haven;t check mmapv1 yet.

      14.59%    mongod  mongod     [.] tcmalloc::ThreadCache::ReleaseToCentralCache(tcmalloc::ThreadCache::FreeList*, unsigned long, int)

      it seems we can adjust max thread cache size to alleviate this situation. The default size is 32M per thread, did test with 64M, which seems improve the throughput

      and here is perf even flame graph, you can check usage interactively. (please download and open in a new window for better view)

        1. thread_cache_64mb_vs_32mb.png
          34 kB
        2. perf-kernel-stable-after-bad3.svg
          403 kB
        3. perf.data.stable-after-bad3.gz
          2.90 MB
        4. perf.data.4000sec.svg
          722 kB
        5. perf.data.4000sec.gz
          8.45 MB
        6. perf.data.22ksec.svg
          626 kB
        7. perf.data.22ksec.gz
          12.43 MB

            geert.bosch@mongodb.com Geert Bosch
            rui.zhang Rui Zhang (Inactive)
            0 Vote for this issue
            8 Start watching this issue