[SERVER-39001] Allow majority read concern for replication for backup node use-case Created: 15/Jan/19 Updated: 06/Dec/22 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | New Feature | Priority: | Major - P3 |
| Reporter: | David Bartley | Assignee: | Backlog - Replication Team |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Assigned Teams: |
Replication
|
| Sprint: | Repl 2019-02-25 |
| Participants: |
| Description |
|
One common configuration is to have a non-voting, hidden backup node. One would then take e.g. hourly filesystem or EBS snapshots of the node. One potential issue with this approach is that it's possible that a backup snapshot might wind up with data that is rolled back, e.g. if the primary fails at the same time a s apshot is taken. One approach to address this is to use delayed replication of e.g. 5 min, and hope that is sufficient to avoid observing the rollback (I'm not sure of the oplog is fixed up during a rollback or not, so perhaps this isn't actually sufficient). Instead, a more robust solution would be to allow the backup node's replication thread/query to use read concern majority. Presumably this would require that you set the node to priority 0/hidden, the same restrictions on delayed replication. |
| Comments |
| Comment by David Bartley [ 23/Apr/20 ] |
|
Another place where this could be useful is to recover from poisoned oplog entries that cause all secondaries to crash loop. |
| Comment by David Bartley [ 05/Feb/20 ] |
|
We'd typically either run a mongod standalone against the snapshot (e.g. to pull historic data, or to validate something in our ETL pipeline), or would bring up a replset with each member running against the snapshot. I don't think we'd ever restore a single node in an existing replset off of a snapshot, though, which I agree would trigger a rollback if necessary. |
| Comment by Alyson Cabral (Inactive) [ 05/Feb/20 ] |
|
bartle Hey Bartle, is this a case where you're using backups for compliance? I just want to clarify that once a backup is restored to a node, that node will undergo rollback if necessary. |
| Comment by Danny Hatcher (Inactive) [ 15/Jan/19 ] |
|
Hello David, Thank you for your report. I have assigned this ticket to our Replication team to take a look. Danny |