Uploaded image for project: 'MongoDB ETL Tools'
  1. MongoDB ETL Tools
  2. TOOLS-1720

mongodump improved behavior with --oplog

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: mongodump
    • Labels:
      None
    • Case:

      Description

      The following enterprise features are sorely missed by customers of the tool

      1. Cannot take a consistent backup of a single database (e.g. using --oplog)
      2. --oplog does not tail the oplog and can fail after a lengthy dump

      mongodump --oplog -vvvvv
      2017-07-05T15:21:29.580-0400	found databases: local, test
      2017-07-05T15:21:29.580-0400	enqueued collection 'test.c'
      2017-07-05T15:21:29.581-0400	determined cluster to be a replica set
      2017-07-05T15:21:29.581-0400	oplog located in local.oplog.rs
      2017-07-05T15:21:29.581-0400	dump phase I: metadata, indexes, users, roles, version
      2017-07-05T15:21:29.581-0400		reading indexes for `test.c`
      2017-07-05T15:21:29.582-0400	determined cluster to be a replica set
      2017-07-05T15:21:29.582-0400	oplog located in local.oplog.rs
      2017-07-05T15:21:29.582-0400	getting most recent oplog timestamp
      2017-07-05T15:21:29.582-0400	dump phase II: regular collections
      2017-07-05T15:21:29.582-0400	finalizing intent manager with longest task first prioritizer
      2017-07-05T15:21:29.582-0400	dumping up to 2 collections in parallel
      2017-07-05T15:21:29.582-0400	starting dump routine with id=1
      2017-07-05T15:21:29.582-0400	will listen for SIGTERM, SIGINT and SIGHUP
      2017-07-05T15:21:29.582-0400	starting dump routine with id=0
      2017-07-05T15:21:29.582-0400	ending dump routine with id=0, no more work to do
      2017-07-05T15:21:29.586-0400	writing test.c to
      2017-07-05T15:21:29.586-0400	counted 1422452 documents in test.c
      2017-07-05T15:21:32.582-0400	[######################..]  test.c  1351210/1422452  (95.0%)
      2017-07-05T15:21:32.664-0400	[########################]  test.c  1432268/1422452  (100.7%)
      2017-07-05T15:21:32.664-0400	done dumping test.c (1432268 documents)
      2017-07-05T15:21:32.666-0400	ending dump routine with id=1, no more work to do
      2017-07-05T15:21:32.666-0400	dump phase III: the oplog
      2017-07-05T15:21:32.666-0400	checking if oplog entry 6439369257720481699 still exists
      2017-07-05T15:21:32.883-0400	oldest oplog entry has timestamp 6439369262015447267
      2017-07-05T15:21:32.883-0400	oldest oplog entry of timestamp 6439369262015447267 is older than 6439369257720481699
      2017-07-05T15:21:32.883-0400	Failed: oplog overflow: mongodump was unable to capture all new oplog entries during execution
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                david.golden David Golden
                Reporter:
                vick.mena Vick Mena
              • Votes:
                5 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: