[SERVER-43000] Unify classes that hold lists of oplog entries Created: 23/Aug/19  Updated: 29/Oct/23  Resolved: 22/Nov/19

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: 4.3.3

Type: Improvement Priority: Major - P3
Reporter: A. Jesse Jiryu Davis Assignee: A. Jesse Jiryu Davis
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Sprint: Repl 2019-12-02
Participants:

 Description   

Today, oplog entries are moved among several lists on their way from OplogFetcher to SyncTail::multiApply:

  • Fetched oplog entries are continuously appended to the OplogBuffer.
  • The OpQueueBatcher's thread moves entries into a batch, which is called an OpQueue but is just a list of oplog entries.
  • The OplogApplier's thread calls SyncTail::_oplogApplication(), which gets the OpQueueBatcher's current OpQueue and moves its vector<OplogEntry> to a different vector<OplogEntry>. (The two vectors are typedef'ed as OplogApplier::Operations and MultiApplier::Operations.)
  • SyncTail::multiApply receives the vector, converts it into a OplogEntryBatch of one or more entries, and passes it to SyncTail::syncApply (directly or via InsertGroup::groupAndApplyInserts)
  • SyncTail (which by now will have been merged into OplogApplierImpl) also passes around lists of type std::vector<MultiApplier::Operation> and std::vector<MultiApplier::OperationPtrs>, see if these can be removed.

The number of steps in the pipeline is probably not a performance problem, but the data flow is confusing, and there are many implementations of the same basic idea of a list of oplog entries.

Rename OpQueue to OplogBatch, and remove all typedefs of vector<OplogEntry>, they obscure more than they help.

Rename OpQueueBatcher to OplogBatcher.

The only remaining list of oplog entries is OplogEntryBatch (renamed OplogEntryOrGroupedInserts in Step 10), which holds one oplog entry or a list of insert oplog entries. This class is useful enough to keep.



 Comments   
Comment by Githook User [ 22/Nov/19 ]

Author:

{'email': 'jesse@mongodb.com', 'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis'}

Message: SERVER-43000 Rename OpQueueBatcher to OplogBatcher
Branch: master
https://github.com/mongodb/mongo/commit/355937f48d5c505d606ac1c211fb6427179d8a5b

Comment by Githook User [ 21/Nov/19 ]

Author:

{'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis', 'email': 'jesse@mongodb.com'}

Message: SERVER-43000 Rename OpQueue to OplogBatch
Branch: master
https://github.com/mongodb/mongo/commit/a29b23f30cfe877160993d0ee1646e28939a1da6

Comment by Githook User [ 21/Nov/19 ]

Author:

{'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis', 'email': 'jesse@mongodb.com'}

Message: SERVER-43000 Remove typedefs for vectors of OplogEntry
Branch: master
https://github.com/mongodb/mongo/commit/2808aa41736d241000f81f1f7147a72a4fbd47c5

Generated at Thu Feb 08 05:01:59 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.