-
Type: Improvement
-
Resolution: Duplicate
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Replication
-
None
-
Repl 2019-09-23, Repl 2019-10-07
Currently oplog application is split between OplogApplier, SyncTail, applyOps, transaction_oplog_application.cpp, and oplog.cpp. The OplogApplier is the direction we want to be moving towards but it wasn't fully completed. We likely will not want to merge everything together, but we will save ourselves a lot of time in the future if we finish the OplogApplier and make oplog application code much more straightforward.
From the epic's design doc, this is the step "Merge SyncTail into OplogApplierImpl":
These two classes are tightly coupled, and don't have a clear separation of concerns. It appears we intended to replace SyncTail with OplogApplierImpl but stopped halfway; let's complete that process.
Both classes:
- Buffer oplog entries from a source
- Delegate to the code that applies the entries
- Delegate to the code that appends entries to the local oplog
OplogApplierImpl has some distinct responsibilities:
- Receive oplog entries via OplogApplier::enqueue()
SyncTail has distinct responsibilities:
- Gather writes into groups and assign them to worker threads in fillWriterVectors() and _applyOps().
- Directly call functions that apply ops and append them to the local oplog.
We will merge SyncTail's methods into OplogApplierImpl incrementally, starting from the root of the call tree:
- SyncTail::oplogApplication. It will have been renamed _runLoop. Also move OpQueueBatcher class to its own file.
- SyncTail::shutdown.
- SyncTail::multiApply and the free function scheduleWritesToOplog.
- SyncTail::fillWriterVectors.
- SyncTail::applyOps and the free function multiSyncApply (rename to applyGroup).
- The static function SyncTail::syncApply. It will have been converted to a free function. Rename it to applyOplogEntryBatch and remove lying comment, "Functions may be overridden for testing". This function has no such overrides.
As a side effect we'll remove the inconsistent "sync" method-naming convention. We'll have the opportunity to review each method's code as we go, improving and updating it.
- depends on
-
SERVER-43327 Move runLoop from SyncTail to OplogApplierImpl and move OpQueueBatcher to its own file
- Closed
-
SERVER-43344 Move shutdown, multiApply, and scheduleWritesToOplog from SyncTail to OplogApplierImpl
- Closed
-
SERVER-43651 Move fillWriterVectors, multiSyncApply, and syncApply to OplogApplierImpl
- Closed
- is depended on by
-
SERVER-42996 Move ApplierState to OplogApplier
- Closed
- is duplicated by
-
SERVER-43327 Move runLoop from SyncTail to OplogApplierImpl and move OpQueueBatcher to its own file
- Closed
-
SERVER-43344 Move shutdown, multiApply, and scheduleWritesToOplog from SyncTail to OplogApplierImpl
- Closed
-
SERVER-43651 Move fillWriterVectors, multiSyncApply, and syncApply to OplogApplierImpl
- Closed