Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-37795

WiredTiger again uses way more memory than instructed by cache limit in 3.6

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Works as Designed
    • Affects Version/s: 3.6.7, 3.6.8
    • Fix Version/s: None
    • Component/s: WiredTiger
    • Operating System:
      ALL
    • Sprint:
      Storage Engines 2018-11-19, Storage Engines 2018-12-03
    • Case:

      Description

      As I said in https://jira.mongodb.org/browse/SERVER-17424?focusedCommentId=2044120&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-2044120 I noticed yet another regression in 3.6 about memory usage this time (after the secondary CPU × 4 increase and the IO usage × 5 increase), I now have almost twice the configured cacheSize used by mongo, and it increased a lot since 3.4.15:

      I first upgraded to 3.6.7 and then 3.6.8 after https://jira.mongodb.org/browse/SERVER-35958?focusedCommentId=2007204&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-2007204

      My configured max cache is 10G (server has 32G RAM):

      rs0:PRIMARY> db.serverStatus().version
      3.6.8
      rs0:PRIMARY> db.serverStatus().wiredTiger.cache["maximum bytes configured"]
      10737418240
      

      And my actual usage is 19G RES (21G VIRT):

      rs0:PRIMARY> db.serverStatus().tcmalloc.tcmalloc.formattedString
      ------------------------------------------------
      MALLOC:    16556355400 (15789.4 MiB) Bytes in use by application
      MALLOC: +   3597598720 ( 3430.9 MiB) Bytes in page heap freelist
      MALLOC: +    227350168 (  216.8 MiB) Bytes in central cache freelist
      MALLOC: +       884736 (    0.8 MiB) Bytes in transfer cache freelist
      MALLOC: +    290449952 (  277.0 MiB) Bytes in thread cache freelists
      MALLOC: +     67035392 (   63.9 MiB) Bytes in malloc metadata
      MALLOC:   ------------
      MALLOC: =  20739674368 (19778.9 MiB) Actual memory used (physical + swap)
      MALLOC: +   1484984320 ( 1416.2 MiB) Bytes released to OS (aka unmapped)
      MALLOC:   ------------
      MALLOC: =  22224658688 (21195.1 MiB) Virtual address space used
      MALLOC:
      MALLOC:         286938              Spans in use
      MALLOC:            886              Thread heaps in use
      MALLOC:           4096              Tcmalloc page size
      ------------------------------------------------
      

      Here is the full serverStatus: server_status.json

        Attachments

        1. excess.png
          excess.png
          166 kB
        2. image-2018-10-29-09-22-50-604.png
          image-2018-10-29-09-22-50-604.png
          17 kB
        3. mongodb-3.6-memory-regression.png
          mongodb-3.6-memory-regression.png
          18 kB
        4. server_status.json
          34 kB

          Issue Links

            Activity

              People

              Assignee:
              alexander.gorrod Alexander Gorrod
              Reporter:
              bigbourin@gmail.com Adrien Jarthon
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              18 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                    PagerDuty