[SERVER-76818] Separate secondary oplog application storage transactions with many index writes Created: 03/May/23 Updated: 01/Dec/23 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Judah Schvimer | Assignee: | Backlog - Replication Team |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Replication
|
||||||||
| Participants: | |||||||||
| Description |
|
If applying an oplog entry requires many index writes, it can cause write conflicts (due to cache pressure) and slowness, especially as the size of the storage transaction increases. There's no need to apply the entire oplog entry in one storage transaction (we currently split up prepared transactions into multiple storage transactions, for example). I'm not sure the best way to determine when to split up a storage transaction. |
| Comments |
| Comment by Louis Williams [ 01/Dec/23 ] |
|
Creating more storage transactions on secondaries will slow down the time it takes to complete a batch, so we should be careful here. Ideally, we would only do this when the storage engine is actually under cache pressure. |