Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-1556

Mongodump memory leak (--gzip --out)

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: 3.2.11
    • Component/s: mongodump
    • Environment:
    • Platforms 2016-11-21, Platforms 2017-01-23

      A mongodump with compression to an output directory (not archive) rapidly consumes all available RAM and is terminated by OOMkiller. Eg. with 8GB data size, mongodump reached ±24GB RSS before being terminated. A few times, there was a stacktrace, which I've attached. Mostly, it's OOMkiller.

      Note:

      • Without the gzip option, the dump completes successfully.
      • `mongodump --gzip --archive=/var/backups/mongodb/test/` completes successfully.
      [ 2603.248664] Out of memory: Kill process 7306 (mongodump) score 730 or sacrifice child
      [ 2603.248790] Killed process 7306 (mongodump) total-vm:29607944kB, anon-rss:24987428kB, file-rss:0kB
      
      # with: time -v mongodump --gzip --out ~/
      Command terminated by signal 9
          User time (seconds): 304.06
          System time (seconds): 151.85
          Percent of CPU this job got: 284%
          Elapsed (wall clock) time (h:mm:ss or m:ss): 2:40.31
          Average shared text size (kbytes): 0
          Average unshared data size (kbytes): 0
          Average stack size (kbytes): 0
          Average total size (kbytes): 0
          Maximum resident set size (kbytes): 24988952
          Average resident set size (kbytes): 0
          Major (requiring I/O) page faults: 290
          Minor (reclaiming a frame) page faults: 6321750
          Voluntary context switches: 1067807
          Involuntary context switches: 135809
          Swaps: 0
          File system inputs: 54280
          File system outputs: 653864
          Socket messages sent: 0
          Socket messages received: 0
          Signals delivered: 0
          Page size (bytes): 4096
          Exit status: 0
      

      The exact command line arguments being supplied:

      mongodump --ssl --sslAllowInvalidCertificates --host hidden-secondary.example.com --port 27017 --username backup --password REDACTED --authenticationDatabase admin --out /var/backups/mongodb/test/ --gzip
      

            Assignee:
            gabriel.russell@mongodb.com Gabriel Russell (Inactive)
            Reporter:
            john.shearar John Shearar
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: