Steps to reproduce:
To reproduce the problem follow these steps
- Start a replica set (so we can dump the oplog)
- Create an index
- Create an initial set of documents
- Start a standalone mongod to send the restore to
- Start walking through the documents removing and inserting
- While this is running start a dump and restore
The restore fails
Doing a backup using mongodump --oplog and verifying the backups by running a restore --oplogReplay causes a unique key violation, either during index build or inserts/oplog-replay. This is due to the fact that multiple points in time of the unique index (not _id index which is used as the default index during backup traversal = snapshot mode) are combined during restore. In replication this happens in recovery mode which changes unique index constraints until the oplog can replay to a consistent point in time.
A reproduction does the following:
- running remove/insert with the same accountNumberId, but different docs
- running mongodump --oplog
- mongorestore --oplogReplay
Depending on if the restore is done from an empty collection (or with --drop) or not will determine where the error occurs during restore (either index will not be built correctly, or bad/wrong data will exist for documents depending on the duplicates – based on the unique index). If the index has the dropDups options then the index creation will not fail but the documents deleted may not be correct in comparison to the orig. data.
Adam Comerford put together the attached diagram.