[SERVER-41882] Finish the OplogApplier Created: 24/Jun/19  Updated: 02/Oct/19  Resolved: 02/Oct/19

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Judah Schvimer Assignee: Mihai Andrei
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-43327 Move runLoop from SyncTail to OplogAp... Closed
depends on SERVER-43344 Move shutdown, multiApply, and schedu... Closed
depends on SERVER-43651 Move fillWriterVectors, multiSyncAppl... Closed
is depended on by SERVER-42996 Move ApplierState to OplogApplier Closed
Duplicate
is duplicated by SERVER-43327 Move runLoop from SyncTail to OplogAp... Closed
is duplicated by SERVER-43344 Move shutdown, multiApply, and schedu... Closed
is duplicated by SERVER-43651 Move fillWriterVectors, multiSyncAppl... Closed
Sprint: Repl 2019-09-23, Repl 2019-10-07
Participants:

 Description   

Currently oplog application is split between OplogApplier, SyncTail, applyOps, transaction_oplog_application.cpp, and oplog.cpp. The OplogApplier is the direction we want to be moving towards but it wasn't fully completed. We likely will not want to merge everything together, but we will save ourselves a lot of time in the future if we finish the OplogApplier and make oplog application code much more straightforward.

From the epic's design doc, this is the step "Merge SyncTail into OplogApplierImpl":

These two classes are tightly coupled, and don't have a clear separation of concerns. It appears we intended to replace SyncTail with OplogApplierImpl but stopped halfway; let's complete that process.

Both classes:

  • Buffer oplog entries from a source
  • Delegate to the code that applies the entries
  • Delegate to the code that appends entries to the local oplog

OplogApplierImpl has some distinct responsibilities:

  • Receive oplog entries via OplogApplier::enqueue()

SyncTail has distinct responsibilities:

  • Gather writes into groups and assign them to worker threads in fillWriterVectors() and _applyOps().
  • Directly call functions that apply ops and append them to the local oplog.

We will merge SyncTail's methods into OplogApplierImpl incrementally, starting from the root of the call tree:

  1. SyncTail::oplogApplication. It will have been renamed _runLoop. Also move OpQueueBatcher class to its own file.
  2. SyncTail::shutdown.
  3. SyncTail::multiApply and the free function scheduleWritesToOplog.
  4. SyncTail::fillWriterVectors.
  5. SyncTail::applyOps and the free function multiSyncApply (rename to applyGroup).
  6. The static function SyncTail::syncApply. It will have been converted to a free function. Rename it to applyOplogEntryBatch and remove lying comment, "Functions may be overridden for testing". This function has no such overrides.

As a side effect we'll remove the inconsistent "sync" method-naming convention. We'll have the opportunity to review each method's code as we go, improving and updating it.


Generated at Thu Feb 08 04:58:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.