Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-2046

mongorestore --oplogReplay is very slow

    • Type: Icon: Task Task
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.4.15
    • Component/s: mongorestore
    • 1,885

      The `–oplogReplay` option for `mongorestore` is very useful as a way to get point-in-time backups. However, in our tests the oplog restore operation is very slow, averaging around 0.5MB/s.

       

      Basic test setup:

      • EC2 r4.large instance
      • EBS volume with 4500 IOPS
      • MongoDB and mongodump 3.4.15.

       

      When restoring a large dump (around 1TB), the database itself is restored in a decent time (around 20 hours, or 13.9MB/s). However, the oplog replay portion is very slow, with only around 4GB of oplog restored another 3 hours later (around 0.5MB/s).

      It seems like the oplog entries are applied individually, and not as a batch operation (my knowledge on this is limited, so I may be wrong). Would it help if the mongorestore command applied these as a batch operation?

       

       

            Assignee:
            Unassigned Unassigned
            Reporter:
            ralf@embarkmobile.com Ralf Kistner
            Votes:
            2 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated: