[SERVER-33866] Decouple oplog fetching from oplog application Created: 13/Mar/18  Updated: 22/Aug/23  Resolved: 22/Aug/23

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Spencer Brody (Inactive) Assignee: Backlog - Replication Team
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-32513 Initial sync unnecessarily throws awa... Closed
Assigned Teams:
Replication
Participants:

 Description   

Currently we don't fetch a new batch of oplog entries until we have finished applying the previous batch. Instead, we should fetch oplog entries as fast as we can get them from our sync source and write them down, and in parallel we should be applying oplog entries as fast as we can. This would be especially useful in conjunction with SERVER-32513, so that when coming out of initial sync you never stop syncing from your sync source, which would greatly limit the degree to which initial sync was dependent on the size of its sync source's oplog.



 Comments   
Comment by Eric Milkie [ 14/Mar/18 ]

Oplog fetching is already decoupled from oplog application. I think what you mean instead is that the act of writing the oplog is currently coupled to the applier applying batches of oplog entries. This behavior used to be necessary when the "end of the oplog" was meaningful, but we have since changed the logic such that the last batch boundary is durably and separately tracked. Thus, we can certainly write oplog entries into the oplog without regard to the progress of the applier.

Generated at Thu Feb 08 04:34:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.