Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-1590

mongodump with --oplog should not count the amount of entries

    • Type: Icon: Task Task
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.2.11
    • Component/s: mongodump
    • 1
    • Not Needed
    • v3.4, v3.2

      Currently, oplog dumping uses the dumpQueryToIntent (previously dumpQueryToWriter) that checks the command-line query to be non-empty. I believe that when oplog is being dumped, this check should be skipped and the total oplog entry count should not be obtained, because it effectively scans the oplog in full (since there is on index on "ts" field and the query is something along the lines of {ts:{$gt:...}}).
      While having the progress bar is certainly helpful, forcing an extra (possibly 100+GB) oplog scan to output it properly seems a bit overkill, since this leads to excessive disk utilization on databases with large historical oplogs (not to mention killing the cache on MMAPv1 engines, straining the database).
      Also, although I'm not sure, maybe the "query" instead of "dump.query" should be checked, then it will work for oplog queries automatically.

            Assignee:
            Unassigned Unassigned
            Reporter:
            onyxmaster Aristarkh Zagorodnikov
            Votes:
            3 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: