[SERVER-12508] Add Replica Set Restore Mode (for point-in-time) Created: 28/Jan/14  Updated: 14/Nov/19  Resolved: 08/Nov/19

Status: Closed
Project: Core Server
Component/s: Admin, Replication
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Scott Hernandez (Inactive) Assignee: Judah Schvimer
Resolution: Won't Fix Votes: 6
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by TOOLS-176 Dump/Restore with --oplog not point-i... Closed
Duplicate
is duplicated by SERVER-10773 Provide mechanism for cloud to reliab... Closed
Related
is related to TOOLS-852 mongorestore --noDataRestore Accepted
is related to SERVER-19618 Allow applyOps to ignore unique index... Closed
is related to TOOLS-2345 Does oplogApplicationMode 'initialSyn... Closed
Operating System: ALL
Sprint: Repl 2019-08-26
Participants:
Linked BF Score: 15

 Description   

This mode/command will need to support situations where unique indexes restraints must be loosened like during initial-sync/recovery mode.

Depending on the state of the data, and the contents of the oplog, in a backup taken with "mongodump --oplog", attempting to restore the data may cause temporary unique-index violations. The violations will be resolved by the time "mongorestore" has replayed the dumped oplog, but it's currently impossible to do the mongorestore because the unique-index violations will abort the restore.

We need a way to suspend unique-index constraints while restoring a dump with an oplog, then re-enable the constraints at the end of the restore. There should be a rigorous strategy in place for dealing with unique-index violations that remain at the end of the restore.



 Comments   
Comment by Judah Schvimer [ 31/Jul/19 ]

The applyOps oplogApplicationMode was added to let applyOps mimic initial sync in all ways. Before we dive into a mini-design can you please test if it already provides the functionality you want? If it does not, would making it ignore index constraints be sufficient for your purposes?

Comment by David Golden [ 31/Jul/19 ]

I don't see anything in that ticket about index constraints, only about FCV changes. I looked at the commit and at the current code base and it didn't seem like that flag would address this, but I admit I'm not very familiar with the code paths in question.

(mongomirror currently doesn't yet use oplogApplicationMode, but I'd be happy to collaborate on a mini-design for using it and what it does.)

Comment by Judah Schvimer [ 31/Jul/19 ]

david.golden, did the "initial sync mode" to applyOps added in SERVER-31507 not accomplish what you are looking for?

Comment by David Golden [ 05/Feb/19 ]

I believe mongomirror works around this by deferring index creation until after oplog replay. Tools could do the same, but that won't help when restoring into an existing database.

Is there – or could there be – an option on applyOps that ignores unique constraint violations for the batch of ops to apply?

Comment by Krzysztof [ 18/Sep/18 ]

Can you provide information when approximatly this bug can be addressed? This issue cause that all backups with oplog replay is compromised...

 

Comment by Asya Kamsky [ 22/Feb/18 ]

skelly I think the primary issue is the _id index which is created as soon as collection is created.

Comment by Seth Kelly [ 22/Feb/18 ]

Dumb question. Couldn't you change the ordering in the restore, and process the oplog before applying indexes?

Generated at Thu Feb 08 03:28:43 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.