Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-2839

Mongorestore high memory usage after upgrade from 3.6 to 4.2

    XMLWordPrintable

    Details

    • Type: Investigation
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: No versions
    • Component/s: mongorestore
    • Labels:
      None
    • Case:

      Description

      Problem Description

      After upgrade from 3.6 to 4.2 (3.6 --> 4.0 --> 4.2), mongorestore allocates over 6GB ram and then start swapping. Consequently, restore is very slow (over 45 minutes instead of 7 minutes with mongo 3.6)

      Steps to Reproduce

       

      mongorestore \ --host localhost:27017 \ -u restore_user --authenticationDatabase "admin" \ -p <secret> \ --gzip \ --dir "${TMPDIR}/rs0/dump" \ --drop
      

      Expected Results

      Actual Results

      Additional Notes

       

       ===== Starting restore procedure: Fri Apr  9 13:23:27 CEST 2021.... 2021-04-09T13:23:39.611+0200    preparing collections to restore from 2021-04-09T13:23:39.652+0200    reading metadata for wf_engine.metadata from /tmp/restore.5gdfVY/rs0/dump/wf_engine/metadata.metadata.json.gz 2021-04-09T13:23:39.661+0200    reading metadata for inventario_be.infn_inv_richiesta from /tmp/restore.5gdfVY/rs0/dump/inventario_be/infn_inv_richiesta.metadata.json.gz 2021-04-09T13:23:39.691+0200    restoring wf_engine.metadata from /tmp/restore.5gdfVY/rs0/dump/wf_engine/metadata.bson.gz 2021-04-09T13:23:39.724+0200    reading metadata for wf_engine.transition from /tmp/restore.5gdfVY/rs0/dump/wf_engine/transition.metadata.json.gz 2021-04-09T13:23:39.726+0200    reading metadata for wf_engine.workflowsInstance from /tmp/restore.5gdfVY/rs0/dump/wf_engine/workflowsInstance.metadata.json.gz 2021-04-09T13:23:39.734+0200    restoring inventario_be.infn_inv_richiesta from /tmp/restore.5gdfVY/rs0/dump/inventario_be/infn_inv_richiesta.bson.gz 2021-04-09T13:23:39.741+0200    restoring wf_engine.workflowsInstance from /tmp/restore.5gdfVY/rs0/dump/wf_engine/workflowsInstance.bson.gz 2021-04-09T13:23:39.777+0200    no indexes to restore 2021-04-09T13:23:39.777+0200    finished restoring inventario_be.infn_inv_richiesta (8 documents, 0 failures) 2021-04-09T13:23:39.782+0200    restoring wf_engine.transition from /tmp/restore.5gdfVY/rs0/dump/wf_engine/transition.bson.gz 2021-04-09T13:23:39.786+0200    reading metadata for wf_engine.workflows from /tmp/restore.5gdfVY/rs0/dump/wf_engine/workflows.metadata.json.gz 2021-04-09T13:23:39.797+0200    restoring wf_engine.workflows from /tmp/restore.5gdfVY/rs0/dump/wf_engine/workflows.bson.gz 2021-04-09T13:23:39.888+0200    no indexes to restore 2021-04-09T13:23:39.888+0200    finished restoring wf_engine.workflows (8 documents, 0 failures) 2021-04-09T13:23:42.627+0200    [........................]           wf_engine.metadata  11.6MB/496MB  (2.3%) 2021-04-09T13:23:42.627+0200    [#.......................]  wf_engine.workflowsInstance  15.7MB/340MB  (4.6%) 2021-04-09T13:23:42.627+0200    [........................]         wf_engine.transition  6.21MB/275MB  (2.3%) …. 2021-04-09T14:09:57.644+0200    [########################]  wf_engine.metadata  496MB/496MB  (100.0%) 2021-04-09T14:09:59.179+0200    [########################]  wf_engine.metadata  496MB/496MB  (100.0%) 2021-04-09T14:09:59.193+0200    restoring indexes for collection wf_engine.metadata from metadata 2021-04-09T14:12:59.179+0200    finished restoring wf_engine.metadata (107 documents, 0 failures) 2021-04-09T14:12:59.191+0200    restoring users from /tmp/restore.5gdfVY/rs0/dump/admin/system.users.bson.gz 2021-04-09T14:12:59.480+0200    restoring roles from /tmp/restore.5gdfVY/rs0/dump/admin/system.roles.bson.gz 2021-04-09T14:13:00.048+0200    422001 document(s) restored successfully. 0 document(s) failed to restore. ===== Finished restore procedure: Fri Apr  9 14:13:02 CEST 2021.... 

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              tim.fogarty Tim Fogarty
              Reporter:
              bovy89@gmail.com bovy89 N/A
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: