[SERVER-70903] support multi-oplog format for batched operations Created: 27/Oct/22 Updated: 02/Feb/24 Resolved: 17/Jan/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 6.3.0-rc0 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Benety Goh | Assignee: | Benety Goh |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||||||||||||||
| Sprint: | Execution Team 2022-11-14, Execution Team 2022-12-12, Execution Team 2022-11-28, Execution Team 2022-12-26, Execution Team 2023-01-09, Execution Team 2023-01-23 | ||||||||||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
Batched operations (see BatchedWriteContext) are currently limited to replicating as a single oplog entry. This places the responsibility of limiting the operations in a batched write on the caller. It would be nice to re-use the chained applyOps format used for large multi-document transactions to format large batched operations in a similar manner. Callers would no longer be constrained by the single applyOps format. However, for performance reasons, callers should still be aware of the implications of allowing unbounded set of operations to be batched and handle these issues accordingly. This ticket is limited to having the primary emit a chain of applyOps oplog entries representing a large batched write. Processing this chain of applyOps oplog entries on the secondary will be addressed in a follow-up ticket. This behavior will be guarded by a feature flag. |
| Comments |
| Comment by Benety Goh [ 23/Feb/23 ] |
|
Batched writes that span multiple oplog entries will now be formatted similarly to large multi-doc transactions with a prevOpTime field linking each oplog entry in the chain to the previous entry. This is a flag-guarded behavior. For backwards compatibility, batched writes that fit within a single oplog entry will omit the prevOpTime field. |
| Comment by Githook User [ 14/Jan/23 ] |
|
Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}Message: |
| Comment by Githook User [ 13/Jan/23 ] |
|
Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}Message: |
| Comment by Githook User [ 13/Jan/23 ] |
|
Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}Message: |
| Comment by Benety Goh [ 11/Jan/23 ] |
|
Supporting multi-oplog format for batched writes will requires relaxing the multi-timestamp constraint similar to what we do for unprepared multi-document transactions. SERVER-51301 describes a longer term strategy for RecoveryUnit::ignoreAllMultiTimestampConstraints(). |