Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-43000

Unify classes that hold lists of oplog entries

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.3.3
    • Affects Version/s: None
    • Component/s: Replication
    • None
    • Fully Compatible
    • Repl 2019-12-02

      Today, oplog entries are moved among several lists on their way from OplogFetcher to SyncTail::multiApply:

      • Fetched oplog entries are continuously appended to the OplogBuffer.
      • The OpQueueBatcher's thread moves entries into a batch, which is called an OpQueue but is just a list of oplog entries.
      • The OplogApplier's thread calls SyncTail::_oplogApplication(), which gets the OpQueueBatcher's current OpQueue and moves its vector<OplogEntry> to a different vector<OplogEntry>. (The two vectors are typedef'ed as OplogApplier::Operations and MultiApplier::Operations.)
      • SyncTail::multiApply receives the vector, converts it into a OplogEntryBatch of one or more entries, and passes it to SyncTail::syncApply (directly or via InsertGroup::groupAndApplyInserts)
      • SyncTail (which by now will have been merged into OplogApplierImpl) also passes around lists of type std::vector<MultiApplier::Operation> and std::vector<MultiApplier::OperationPtrs>, see if these can be removed.

      The number of steps in the pipeline is probably not a performance problem, but the data flow is confusing, and there are many implementations of the same basic idea of a list of oplog entries.

      Rename OpQueue to OplogBatch, and remove all typedefs of vector<OplogEntry>, they obscure more than they help.

      Rename OpQueueBatcher to OplogBatcher.

      The only remaining list of oplog entries is OplogEntryBatch (renamed OplogEntryOrGroupedInserts in Step 10), which holds one oplog entry or a list of insert oplog entries. This class is useful enough to keep.

            Assignee:
            jesse@mongodb.com A. Jesse Jiryu Davis
            Reporter:
            jesse@mongodb.com A. Jesse Jiryu Davis
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: