[SERVER-73131] clarify cached partial transaction ops state in oplog applier when aborting a prepared transaction Created: 20/Jan/23  Updated: 29/Oct/23  Resolved: 23/Jan/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.3.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Benety Goh Assignee: Benety Goh
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-39438 Write "abort" oplog entry when aborti... Closed
is related to SERVER-41157 Remove abortGuard in commitUnprepared... Closed
is related to SERVER-43651 Move fillWriterVectors, multiSyncAppl... Closed
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2023-01-23
Participants:

 Description   

TLDR: enforce expectations on partialTxnOps in OplogApplierImpl::_deriveOpsAndFillWriterVectors().

The function OplogApplierImpl::_deriveOpsAndFillWriterVectors() is used during oplog application to extract embedded operations from transaction oplog entries (ie. applyOps) from the current batch of operations returned from the oplog batching/fetching logic. The partialTxnOps map caches operations for multi-oplog transactions within the current batch - partial transaction operations outside the current batch in the large transaction is retrieved from the oplog collection in _addOplogChainOpsToWriterVectors().

The batch processing logic currently clears the cached partial transaction operations when it encounters an abort transaction oplog entry in the same batch. Per current rules around oplog batch boundaries, especially with regards to OplogBatch::mustProcessIndividually()|https://github.com/mongodb/mongo/blob/1947ddfa0a6418ea1aae5fa720ffb43aa4b3a4e1/src/mongo/db/repl/oplog_batcher.cpp#L132], an abort transaction oplog entry must be the only operation in any oplog batch.

It would be nice to clarify the logic in the oplog applier, preferably with an invariant on the expected state of partialTxnOps, when we attempt to process an abort for a prepared transaction.



 Comments   
Comment by Githook User [ 20/Jan/23 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-73131 add invariant for abortTransaction and cached partial transaction operations in the same batch
Branch: master
https://github.com/mongodb/mongo/commit/d4c9920963d4475624eb58fc62ca0ce61f1f0846

Comment by Githook User [ 20/Jan/23 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-73131 add test for preparing and aborting a multi-oplog transaction in the same batch

Note this unit test covers a scenario currently disallowed under current oplog batching rules.
Branch: master
https://github.com/mongodb/mongo/commit/da442eb11a61021e06237f7f0f5d4f30bd97d825

Comment by Jason Chan [ 20/Jan/23 ]

Copying from my conversation with Benety.

From what I remember:
We will buffer the partialTxn ops in a memory buffer as part of secondary transaction until we either 1. apply a prepareTxn entry or 2. apply a terminal apply op (an apply op entry which indicates we should commit the unprepared txn).
To answer your question, I don't think we should expect to ever have existing map entries in partialTxnOps when processing an abortTransaction entry at the step you are referencing because:

  • if unprepared txns are aborted, they will not generate an oplog entry and will have been aborted on the primary. So the secondary shouldn't see an abortOplog entry for unprepared transactions at all.
  • If a prepared txn is aborted, the partialTxns list should have already been applied and cleared as part of the secondary applying the prepareTxn entry.

For extra context, one can refer to the design of packing oplog formats for large transactions.

Generated at Thu Feb 08 06:23:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.