Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-33296

Excessive memory usage due to heap fragmentation

    XMLWordPrintable

    Details

    • Operating System:
      Linux
    • Steps To Reproduce:
      Hide

      David Daly - Handing over to you per e-mail discussion.

      Show
      David Daly - Handing over to you per e-mail discussion.
    • Sprint:
      Dev Tools 2019-05-06, Dev Tools 2019-04-22
    • Case:

      Description

      The changes described in SERVER-20306 eliminated a common source of memory fragmentation, but it can still occur for other reasons. Here's an example from a node undergoing initial sync:

      Over time

      • allocated memory never exceeds 8 GB
      • but heap size and resident memory reach nearly 14 GB
      • this is due to an accumulation of pageheap_free_bytes

      A common cause of this is a shifting distribution of allocated memory sizes, which leaves free pages dedicated to one size of buffer unable to be used for new memory requests because they are for a different size buffer.

      Setting TCMALLOC_AGGRESSIVE_DECOMMIT can address this issue by causing tcmalloc to aggressively return the free pages to the o/s where they can then be re-used by tcmalloc to satisfy new memory requests. However this has an unacceptable negative performance impact. Is there a tweak to tcmalloc that can give us better behavior for workloads like this?

        Attachments

        1. fragmentation.png
          fragmentation.png
          104 kB
        2. fragmentation-3.6.2.png
          fragmentation-3.6.2.png
          107 kB

          Issue Links

            Activity

              People

              • Votes:
                2 Vote for this issue
                Watchers:
                26 Start watching this issue

                Dates

                • Created:
                  Updated: