[SERVER-76818] Separate secondary oplog application storage transactions with many index writes Created: 03/May/23  Updated: 01/Dec/23

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Judah Schvimer Assignee: Backlog - Replication Team
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-76299 Report writeConflicts in serverStatus... Closed
Assigned Teams:
Replication
Participants:

 Description   

If applying an oplog entry requires many index writes, it can cause write conflicts (due to cache pressure) and slowness, especially as the size of the storage transaction increases. There's no need to apply the entire oplog entry in one storage transaction (we currently split up prepared transactions into multiple storage transactions, for example). I'm not sure the best way to determine when to split up a storage transaction.



 Comments   
Comment by Louis Williams [ 01/Dec/23 ]

Creating more storage transactions on secondaries will slow down the time it takes to complete a batch, so we should be careful here. Ideally, we would only do this when the storage engine is actually under cache pressure.

Generated at Thu Feb 08 06:33:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.