[SERVER-36492] Reconstruct prepared transactions at the end of initial sync Created: 07/Aug/18  Updated: 29/Oct/23  Resolved: 06/May/19

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: 4.1.11

Type: Task Priority: Major - P3
Reporter: Judah Schvimer Assignee: Samyukta Lanka
Resolution: Fixed Votes: 0
Labels: open_todo_in_code, prepare_initial_sync
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-35879 Add support for reconstituting transa... Closed
depends on SERVER-40018 Remove ServerTransactionsMetrics::get... Closed
is depended on by SERVER-40041 block prepared transactions behind in... Closed
Related
is related to SERVER-39795 Make sure initial sync starts to sync... Closed
Backwards Compatibility: Fully Compatible
Sprint: Repl 2019-01-28, Repl 2019-02-11, Repl 2019-02-25, Repl 2019-03-11, Repl 2019-03-25, Repl 2019-04-08, Repl 2019-04-22, Repl 2019-05-06, Repl 2019-05-20
Participants:
Linked BF Score: 0

 Comments   
Comment by Githook User [ 06/May/19 ]

Author:

{'name': 'Samy Lanka', 'username': 'lankas', 'email': 'samy.lanka@mongodb.com'}

Message: SERVER-36492 Reconstruct prepared transactions at the end of initial sync
Branch: master
https://github.com/mongodb/mongo/commit/8ad1effc48ac5193c5f57630d1fbce8bda0cfdaf

Comment by Louis Williams [ 18/Mar/19 ]

This may require ignoring prepare conflicts during initial sync due to the issues described inĀ SERVER-40167.

Comment by Eric Milkie [ 01/Mar/19 ]

On top of this work, I'm going to need to add some extra logic to complete the implementation of SERVER-38588.

Comment by Samyukta Lanka [ 22/Feb/19 ]

While implementing this ticket it came up that reconstructing a prepared transaction during initial sync will fail in WiredTiger because we try preparing a transaction before the oldest timestamp. During initial sync we set/advance the oldest timestamp after applying each batch during oplog application. We have two potential solutions to this problem:

  1. Prepare transactions at the initialDataTimestamp during initial sync instead of the original prepareTimestamp. This will require that we also change the corresponding commitTransaction timestamps (otherwise the commitTimestamp could be before the prepareTimestamp). This can be done by changing the commitTimestamp if is before the initialDataTimestamp to be after the initialDataTimestamp.
  2. Stop setting the oldestTimestamp during initial sync. To ease the increased cache pressure that this would cause, we'd need to also change writes in initial sync to be untimestamped. The only writes that would be timestamped would be prepareTransaction, but it is okay in this case because those will have to remain in history anyways.

We are currently going to go with solution 1 because it is a more constricted solution that aligns directly with the work from this ticket. Solution 2 is a bigger change that comes with larger risks, but also potentially more long term benefits. We can implement solution 2 in the future if we'd like to make that improvement.

Comment by Siyuan Zhou [ 11/Sep/18 ]

After finishing this work, please also unblacklist the tests in replica_sets_initsync_jscore_passthrough.yml and replica_sets_initsync_static_jscore_passthrough.yml.

Generated at Thu Feb 08 04:43:16 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.