Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-20326

Record "apply" batch boundaries during replication

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Critical - P2
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1.9
    • Component/s: Replication
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Sprint:
      RPL A (10/09/15)
    • Linked BF Score:
      0

      Description

      Currently we record "minValid" when applying each back during replication as a marker of the (minimum) valid point we must reach to allow reads and to transition out of startup/recovery to secondary/primary. This approach works fine as long as we always apply write to all database and insert the ordered oplog entries as the last step in an atomic write operation.

      The task here is to record both the start and end of each batch so the can do writes in the batch in any order needed. For some storage engines combining writes across database, including recording the oplog entries, performs better and can be optimized but doesn't work with the current "minValid" safety design.

      Recording both the start and end will allow us to just apply those entries in the case of failure during recovery, or as a indication that replication has not finished.

      When the batch apply is successfully done we will (logically) reset the value to just the end (or "minValid") to indicate completion and to be backwards compatible (to allow for downgrading).

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              scotthernandez Scott Hernandez
              Reporter:
              scotthernandez Scott Hernandez
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: