Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-41882

Finish the OplogApplier

    • Type: Icon: Improvement Improvement
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Replication
    • None
    • Repl 2019-09-23, Repl 2019-10-07

      Currently oplog application is split between OplogApplier, SyncTail, applyOps, transaction_oplog_application.cpp, and oplog.cpp. The OplogApplier is the direction we want to be moving towards but it wasn't fully completed. We likely will not want to merge everything together, but we will save ourselves a lot of time in the future if we finish the OplogApplier and make oplog application code much more straightforward.

      From the epic's design doc, this is the step "Merge SyncTail into OplogApplierImpl":

      These two classes are tightly coupled, and don't have a clear separation of concerns. It appears we intended to replace SyncTail with OplogApplierImpl but stopped halfway; let's complete that process.

      Both classes:

      • Buffer oplog entries from a source
      • Delegate to the code that applies the entries
      • Delegate to the code that appends entries to the local oplog

      OplogApplierImpl has some distinct responsibilities:

      • Receive oplog entries via OplogApplier::enqueue()

      SyncTail has distinct responsibilities:

      • Gather writes into groups and assign them to worker threads in fillWriterVectors() and _applyOps().
      • Directly call functions that apply ops and append them to the local oplog.

      We will merge SyncTail's methods into OplogApplierImpl incrementally, starting from the root of the call tree:

      1. SyncTail::oplogApplication. It will have been renamed _runLoop. Also move OpQueueBatcher class to its own file.
      2. SyncTail::shutdown.
      3. SyncTail::multiApply and the free function scheduleWritesToOplog.
      4. SyncTail::fillWriterVectors.
      5. SyncTail::applyOps and the free function multiSyncApply (rename to applyGroup).
      6. The static function SyncTail::syncApply. It will have been converted to a free function. Rename it to applyOplogEntryBatch and remove lying comment, "Functions may be overridden for testing". This function has no such overrides.

      As a side effect we'll remove the inconsistent "sync" method-naming convention. We'll have the opportunity to review each method's code as we go, improving and updating it.

            Assignee:
            mihai.andrei@mongodb.com Mihai Andrei
            Reporter:
            judah.schvimer@mongodb.com Judah Schvimer
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: