[SERVER-33292] Have storage dictate where replication recovery should begin playing oplog from Created: 13/Feb/18  Updated: 29/Oct/23  Resolved: 12/Mar/18

Status: Closed
Project: Core Server
Component/s: Replication, Storage
Affects Version/s: None
Fix Version/s: 3.7.3

Type: Task Priority: Major - P3
Reporter: Daniel Gottlieb (Inactive) Assignee: Judah Schvimer
Resolution: Fixed Votes: 0
Labels: rollback-functional
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-30464 Edit startup warning when running rep... Closed
is depended on by SERVER-33348 Remove checkpoint timestamp collection Closed
is depended on by SERVER-33349 Add command to get stable checkpoint ... Closed
Duplicate
is duplicated by SERVER-32304 Amend ReplicationRecovery code to ref... Closed
Related
related to SERVER-47844 Update _setStableTimestampForStorage ... Closed
Backwards Compatibility: Fully Compatible
Sprint: Repl 2018-02-26, Repl 2018-03-12, Repl 2018-03-26
Participants:

 Description   

Replication recovery starts replaying oplog based on the `checkpointTimestamp` document. However, the value of that document may be stale relative to the time of the actual checkpoint.

This leads to two problems:

  1. Replication recovery has to relax constraints when possibly re-applying entries from an earlier time than the data currently represents. The system does not currently have a way to distinguish between "being applied for the first time" and "maybe being applied a subsequent time". Relaxing these constraints brings risk in breaking the typical execution path.
  2. Reading from a timestamp between the stale checkpoint timestamp value and the actual data time can result in incorrect results. Furthermore, it's impossible to know the actual data time, so the last op applied at recovery is the earliest read time that can be correctly satisfied.

The work to be done for this ticket:

  1. Have `recoverToStableTimestamp` return a `StatusWith<Timestamp>`. This is to be used by rollback to determine where to start replication recovery from. Amend KVStorageEngine/KVEngine/WiredTigerKVEngine to satisfy the API type.
  2. Add a method to the storage engine interface that returns the "data timestamp" on disk. This is to be used at startup to determine where to start replication recovery from.
  3. Have replication recovery query/use these timestamp values for determining where to start applying oplog entries from.


 Comments   
Comment by Githook User [ 12/Mar/18 ]

Author:

{'email': 'judah@mongodb.com', 'name': 'Judah Schvimer', 'username': 'judahschvimer'}

Message: SERVER-33292 Have storage dictate where replication recovery should begin playing oplog from
Branch: master
https://github.com/mongodb/mongo/commit/b1102c617e04ff751d702435f9d4521727e579e1

Generated at Thu Feb 08 04:32:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.