[SERVER-35811] Pin stable timestamp behind the oldest prepareTimestamp Created: 26/Jun/18  Updated: 29/Oct/23  Resolved: 06/Nov/18

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.1.5

Type: Task Priority: Major - P3
Reporter: Gregory McKeon (Inactive) Assignee: Pavithra Vetriselvan
Resolution: Fixed Votes: 0
Labels: prepare_errors
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-35873 Maintain the oldest oplog entry times... Closed
is depended on by SERVER-36782 WT erring because commit timestamp is... Closed
is depended on by SERVER-35877 Secondaries commit transactions when ... Closed
is depended on by SERVER-36023 Try to enable replica set transaction... Closed
Related
is related to SERVER-38302 Committing or aborting prepared trans... Closed
Backwards Compatibility: Fully Compatible
Sprint: Repl 2018-08-27, Repl 2018-09-10, Repl 2018-09-24, Repl 2018-10-08, Repl 2018-10-22, Repl 2018-11-05, Repl 2018-11-19
Participants:
Linked BF Score: 56

 Description   

In order to complete this ticket, we will first need to build on the work done in SERVER-35873. First, we should preserve OpTimes instead of Timestamps since we calculate the stable timestamp using OpTimes. Next, we want to keep a separate data structure that tracks all "oldest oplog entry OpTimes" for each transaction whose corresponding abort/commit oplog entry has not been majority committed. To do this, when we remove an "oldest active OpTime" from oldestActiveOplogEntryOpTimes, we will also add it to oldestActiveUncommittedOpTimes along with its corresponding "finishOpTime" (which is the OpTime of the commit/abort oplog entry).

oldestActiveUncommittedOpTimes will be a set of OpTime pairs -> (oldest active OpTime per transaction, corresponding commit/abort oplog entry OpTime).

This should be enough information to set the stable timestamp back to the "oldest active OpTime of transactions whose corresponding commit/abort oplog entries have not been majority committed". Let's refer to this as the "oldest active uncommitted txn OpTime" for now. In ReplicationCoordinatorImpl, we have a function that calculates the stable OpTime. Instead of just calculating the min of the "all committed timestamp" and the current "commit point," we will also add the "oldest active uncommitted txn Timestamp" to this comparison and take the min of all three.

The last thing we need to do is properly remove an OpTime pair once the commit point advances past an entry's commit/abort oplog entry. This would mean that the entry is majority committed and we are able to move the stable timestamp.

Since this code will be separate from the code maintaining the oldestActiveOpTime, it should be simpler to remove once we no longer need to hold the stable timestamp back.



 Comments   
Comment by Githook User [ 01/Apr/19 ]

Author:

{'email': 'benety@mongodb.com', 'name': 'Benety Goh', 'username': 'benety'}

Message: SERVER-35811 fix lint
Branch: master
https://github.com/mongodb/mongo/commit/83b3ca680119f3bfd991233ddf8b546b9c655949

Comment by Githook User [ 01/Apr/19 ]

Author:

{'email': 'pvselvan@umich.edu', 'name': 'Pavi Vetriselvan', 'username': 'pvselvan'}

Message: Revert "SERVER-35811 disallow committing at the prepareTimestamp and pin stable TS before oldest uncommitted TS"

This reverts commit b58420e57a2149c350ebf82815fe708850402296 and 33ac1afd4079e04d12554f9b79d1ab07426caf59.
Branch: master
https://github.com/mongodb/mongo/commit/e433a5aee915568cf73b05e89597903855ed1952

Comment by Pavithra Vetriselvan [ 11/Dec/18 ]

In addition to the two commits for this ticket, we will also have to revert the following tickets after we allow committing before the stable timestamp:
SERVER-38302

Comment by Githook User [ 08/Nov/18 ]

Author:

{'name': 'Pavi Vetriselvan', 'email': 'pvselvan@umich.edu', 'username': 'pvselvan'}

Message: SERVER-35811 disallow committing at the prepareTimestamp
Branch: master
https://github.com/mongodb/mongo/commit/b58420e57a2149c350ebf82815fe708850402296

Comment by Githook User [ 06/Nov/18 ]

Author:

{'name': 'Pavi Vetriselvan', 'email': 'pvselvan@umich.edu', 'username': 'pvselvan'}

Message: Revert "SERVER-35811 disallow committing at the prepareTimestamp"

This reverts commit 2de26d2c3c7f73cc49126ba32402c0a380c8f882.
Branch: master
https://github.com/mongodb/mongo/commit/5794e35f915161d9f556844bd4540273fe4450f7

Comment by Githook User [ 06/Nov/18 ]

Author:

{'name': 'Pavi Vetriselvan', 'email': 'pvselvan@umich.edu', 'username': 'pvselvan'}

Message: SERVER-35811 disallow committing at the prepareTimestamp
Branch: master
https://github.com/mongodb/mongo/commit/2de26d2c3c7f73cc49126ba32402c0a380c8f882

Comment by Githook User [ 30/Oct/18 ]

Author:

{'name': 'Pavi Vetriselvan', 'email': 'pvselvan@umich.edu', 'username': 'pvselvan'}

Message: SERVER-35811 pin stable timestamp behind oldest uncommitted timestamp
Branch: master
https://github.com/mongodb/mongo/commit/33ac1afd4079e04d12554f9b79d1ab07426caf59

Generated at Thu Feb 08 04:41:05 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.