[SERVER-39436] Keep batch size accurate for oplog application of large unprepared transactions Created: 07/Feb/19  Updated: 29/Oct/23  Resolved: 08/Apr/19

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: 4.1.10

Type: Task Priority: Major - P3
Reporter: Siyuan Zhou Assignee: Benety Goh
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-39707 Keep the batch size accurate by inclu... Closed
Backwards Compatibility: Fully Compatible
Sprint: Storage NYC 2019-04-08, Storage NYC 2019-04-22
Participants:

 Description   

Since there might be many operations in the same transaction, the oplog batcher needs to count in-transaction operations towards the batch size for “commit” of unprepared transaction, to keep the batch size of actually applied operations more accurate. In contrast, “prepare” and “commit” for prepared transactions are always applied in their own batches.

We will count the number of operations by keeping the total number of operations in the “commitTransaction” command. The count doesn’t include the commit itself. With the new “count” field, the “o” field of commitTransaction oplog entry looks like:

{ "commitTransaction": 1, "prepared": false, "count": 2 }

Operations in the same transaction must be applied in the same batch, even if the transaction size exceeds the current batch size. When this happens, the transaction can be scheduled into the next batch if it’s not the first operation in the batch.



 Comments   
Comment by Githook User [ 08/Apr/19 ]

Author:

{'name': 'Benety Goh', 'username': 'benety', 'email': 'benety@mongodb.com'}

Message: SERVER-39436 check batch limits on large unprepared transactions using embedded operation count
Branch: master
https://github.com/mongodb/mongo/commit/d4f2108cf99e6c4a3a93f6f71b0aa7aefa9ee353

Comment by Githook User [ 08/Apr/19 ]

Author:

{'email': 'benety@mongodb.com', 'name': 'Benety Goh', 'username': 'benety'}

Message: SERVER-39436 fix batch limit checking in OplogApplier::getNextApplierBatch()
Branch: master
https://github.com/mongodb/mongo/commit/f68f8b7ccb70de47eacbd0d3f196d1de9b6f6b0d

Generated at Thu Feb 08 04:52:03 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.