[SERVER-35879] Add support for reconstituting transactions in their correct state from the transaction table during startup recovery Created: 28/Jun/18 Updated: 29/Oct/23 Resolved: 14/Dec/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Replication |
| Affects Version/s: | None |
| Fix Version/s: | 4.1.7 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Gregory McKeon (Inactive) | Assignee: | Samyukta Lanka |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | prepare_durability, todo_in_code | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||
| Sprint: | Repl 2018-09-10, Repl 2018-10-08, Repl 2018-10-22, Repl 2018-11-05, Repl 2018-11-19, Repl 2018-12-03, Repl 2018-12-17, Repl 2019-01-14 | ||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Comments |
| Comment by Githook User [ 14/Dec/18 ] |
|
Author: {'username': 'lankas', 'email': 'samy.lanka@mongodb.com', 'name': 'Samy Lanka'}Message: |
| Comment by Githook User [ 13/Dec/18 ] |
|
Author: {'username': 'lankas', 'email': 'samy.lanka@mongodb.com', 'name': 'Samy Lanka'}Message: |
| Comment by Githook User [ 23/Oct/18 ] |
|
Author: {'name': 'Samy Lanka', 'email': 'samy.lanka@mongodb.com', 'username': 'lankas'}Message: |
| Comment by Githook User [ 09/Oct/18 ] |
|
Author: {'name': 'Samy Lanka', 'email': 'samy.lanka@mongodb.com', 'username': 'lankas'}Message: |
| Comment by Judah Schvimer [ 03/Oct/18 ] |
|
LGTM with 2 edits. |
| Comment by Siyuan Zhou [ 03/Oct/18 ] |
|
LGTM. For abort, I guess you want to add the logic into the abortTransaction command application. |
| Comment by Samyukta Lanka [ 03/Oct/18 ] |
|
I'm planning on splitting this ticket into a few different commits to simplify the code review process: 1. Dealing with the abort and prepare oplog entries during startup recovery. Each of these will include writing the oplog entries and modifying config.transactions. My plan is to add separate functions to apply_ops.cpp similar to what was added here and call them from applyOps like here. These changes will be tested via unit tests. 2. Dealing with commit oplog entries. This will involve writing the oplog entries, modifying config.transactions and traversing the oplog for the corresponding prepare oplog entry to apply all the operations in the transaction. One option suggested by siyuan.zhou for how to apply all the operations from the transaction is to transform the prepare oplog entry into a normal applyOps entry and use the existing applyOps function. This will also require getting the stable timestamp from replication_recovery.cpp and passing it down to the applier so that we can compare the stable timestamp with the commit oplog entry timestamp to know whether we need to apply all the operations from the transaction or not. These changes will be tested via unit tests and some jstests. 3. Reconstructing the prepared transactions at the end of recovery. This will involve iterating over all entries in config.transactions and for each transaction in prepare, getting the corresponding oplog entry, checking out the session, applying the operations and then preparing the transaction. These changes will be tested mainly via jstests although I may not be able to unblacklist prepare tests from passthrough suites that involve unclean shutdowns, since those sometimes trigger rollbacks. judah.schvimer and siyuan.zhou let me know if you have any comments on this. |
| Comment by Siyuan Zhou [ 11/Sep/18 ] |
|
judah.schvimer and samy.lanka, I assume this is for startup recovery. I'm referring to this ticket when blacklisting tests in replica_sets_kill_secondaries_jscore_passthrough. After finishing this ticket, please unblacklist them. |
| Comment by Judah Schvimer [ 13/Aug/18 ] |
|
I expect this will use a lot of the same machinery as |