Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-2046

mongorestore --oplogReplay is very slow

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Accepted
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: 3.4.15
    • Fix Version/s: None
    • Component/s: mongorestore
    • Labels:

      Description

      The `–oplogReplay` option for `mongorestore` is very useful as a way to get point-in-time backups. However, in our tests the oplog restore operation is very slow, averaging around 0.5MB/s.

       

      Basic test setup:

      • EC2 r4.large instance
      • EBS volume with 4500 IOPS
      • MongoDB and mongodump 3.4.15.

       

      When restoring a large dump (around 1TB), the database itself is restored in a decent time (around 20 hours, or 13.9MB/s). However, the oplog replay portion is very slow, with only around 4GB of oplog restored another 3 hours later (around 0.5MB/s).

      It seems like the oplog entries are applied individually, and not as a batch operation (my knowledge on this is limited, so I may be wrong). Would it help if the mongorestore command applied these as a batch operation?

       

       

        Attachments

          Activity

            People

            Assignee:
            backlog-server-tools Backlog - Server Tools Team
            Reporter:
            ralf@embarkmobile.com Ralf Kistner
            Votes:
            1 Vote for this issue
            Watchers:
            9 Start watching this issue

              Dates

              Created:
              Updated: