[SERVER-20326] Record "apply" batch boundaries during replication Created: 09/Sep/15  Updated: 08/Jan/16  Resolved: 01/Oct/15

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: 3.1.9

Type: Improvement Priority: Critical - P2
Reporter: Scott Hernandez (Inactive) Assignee: Scott Hernandez (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-18983 Process oplog inserts, and applying, ... Closed
Related
is related to SERVER-20466 Rollback and batch apply can each set... Closed
Backwards Compatibility: Fully Compatible
Sprint: RPL A (10/09/15)
Participants:
Linked BF Score: 0

 Description   

Currently we record "minValid" when applying each back during replication as a marker of the (minimum) valid point we must reach to allow reads and to transition out of startup/recovery to secondary/primary. This approach works fine as long as we always apply write to all database and insert the ordered oplog entries as the last step in an atomic write operation.

The task here is to record both the start and end of each batch so the can do writes in the batch in any order needed. For some storage engines combining writes across database, including recording the oplog entries, performs better and can be optimized but doesn't work with the current "minValid" safety design.

Recording both the start and end will allow us to just apply those entries in the case of failure during recovery, or as a indication that replication has not finished.

When the batch apply is successfully done we will (logically) reset the value to just the end (or "minValid") to indicate completion and to be backwards compatible (to allow for downgrading).



 Comments   
Comment by Githook User [ 06/Oct/15 ]

Author:

{u'username': u'scotthernandez', u'name': u'Scott Hernandez', u'email': u'scotthernandez@gmail.com'}

Message: SERVER-20326: Make sure to update the existing minvalid doc if it exists
Branch: master
https://github.com/mongodb/mongo/commit/7f62960716b37662070ba6fada747c60f5968c09

Comment by Githook User [ 02/Oct/15 ]

Author:

{u'username': u'scotthernandez', u'name': u'Scott Hernandez', u'email': u'scotthernandez@gmail.com'}

Message: SERVER-20326: fix apply batch boundaries test
Branch: master
https://github.com/mongodb/mongo/commit/b8cc9df055c727e4afbb730d61e73420954db85a

Comment by Githook User [ 01/Oct/15 ]

Author:

{u'username': u'scotthernandez', u'name': u'Scott Hernandez', u'email': u'scotthernandez@gmail.com'}

Message: SERVER-20326: simplify apply batch boundaries test
Branch: master
https://github.com/mongodb/mongo/commit/c757c1537b348dedf837aaf6458f7c55506bfef2

Comment by Githook User [ 25/Sep/15 ]

Author:

{u'username': u'scotthernandez', u'name': u'Scott Hernandez', u'email': u'scotthernandez@gmail.com'}

Message: SERVER-20326: record apply batch boundaries
Branch: master
https://github.com/mongodb/mongo/commit/18de2f3d6bb4c01fb600bbb0e3a2c2e4ab0cc9ee

Generated at Thu Feb 08 03:53:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.