[SERVER-58410] Discontinue writing to appliedThrough/minValid as part of secondary batch application Created: 10/Jul/21  Updated: 29/Oct/23  Resolved: 18/Jan/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.3.0

Type: Improvement Priority: Major - P3
Reporter: Daniel Gottlieb (Inactive) Assignee: Lingzhi Deng
Resolution: Fixed Votes: 0
Labels: TechnicalDebt, tech-debt, techdebt
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by WT-7712 commit and durable timestamps should ... Closed
Duplicate
is duplicated by SERVER-53642 Cleanup writes to appliedThrough that... Closed
Problem/Incident
causes SERVER-63118 Wait for writes to become durable bef... Closed
Related
related to SERVER-61409 Investigate the cause of invalidated ... Closed
related to SERVER-61583 Add restart test for retryable intern... Closed
related to SERVER-62715 Architecture Guide updates for minVal... Closed
related to SERVER-62745 Clear appliedThrough after startupRec... Closed
related to SERVER-60037 Enable the ordered timestamp assertio... Closed
related to SERVER-69055 Update stale comments regarding minValid Closed
is related to SERVER-61703 Complete TODO listed in SERVER-53642 Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v5.0
Sprint: Replication 2021-11-29, Replication 2021-12-13, Replication 2021-12-27, Replication 2022-01-10, Replication 2022-01-24
Participants:
Linked BF Score: 170

 Description   

appliedThrough/minValid are for a crash recovery mechanism that allowed secondaries to apply oplog entries in parallel in back in a time when every write was journaled.

Modern MongoDB with a standard configuration (WT + stable checkpoints) have no need for these values. appliedThrough has been replaced with the stable checkpoints timestamp (recoveryTimestamp). minValid is obsoleted as we only checkpoint at a timestamp where all previous writes have been committed.

Writing to appliedThrough + minValid for each batch can have a noticeable performance cost in addition to breaking a WT contract (albeit in a benign way, it causes some maintainability problems with WT). We must(ish) choose a timestamp when writing to this document, and that timestamp is often the stable timestamp.

But for clarity, I think the values should stay and still the server should maintain existing behavior when those values exist. I believe backup/restore relies on them as well as maybe initial sync and rollback via refetch?

The complete ask:

  • Cease the writes before and after every batch of secondary oplog application
  • Never write to it with a timestamp. Or always? With more thoughtful consideration of the stable timestamp.
    • With care. I haven't combed through the more nuanced writes to this document myself to understand any possible complications.


 Comments   
Comment by Githook User [ 18/Jan/22 ]

Author:

{'name': 'Lingzhi Deng', 'email': 'lingzhi.deng@mongodb.com', 'username': 'ldennis'}

Message: SERVER-58410: Remove no stable recovery timestamp invariant after initial sync
Branch: master
https://github.com/mongodb/mongo/commit/f857a8efec9cde7a8c6ee903043e2cd4b5396d48

Comment by Githook User [ 15/Jan/22 ]

Author:

{'name': 'Lingzhi Deng', 'email': 'lingzhi.deng@mongodb.com', 'username': 'ldennis'}

Message: SERVER-58410: Make minValid writes untimestamped
Branch: master
https://github.com/mongodb/mongo/commit/e60858dc5165457c7c5f8574af8d2dead06143d0

Comment by Daniel Gottlieb (Inactive) [ 14/Oct/21 ]

cc louis.williams, yuhong.zhang, I believe most paths to removing mixed-mode writes on this collection also result in resolving this ticket. I'll refrain from making a separate ticket that asks to remove mixed-mode writes on local.replset.minvalid to achieve SERVER-60037 as I wouldn't be able to say right now what this new ticket would need to do that's different than what I'd like to see from this one.

Generated at Thu Feb 08 05:44:26 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.