[SERVER-84706] Investigate if setting the oldest timestamp greater than the stable timestamp can be avoided Created: 09/Jan/24  Updated: 07/Feb/24

Status: In Code Review
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Critical - P2
Reporter: Etienne Petrel Assignee: Xuerui Fa
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-80123 Enable background compaction by defau... Open
is depended on by SERVER-85406 Document more clearly the difference ... Blocked
is depended on by WT-12032 Investigate an out-of-order timestamp... Closed
Gantt Dependency
has to be done before SERVER-80123 Enable background compaction by defau... Open
Related
related to SERVER-85688 Set stable timestamp to end of each o... Open
related to SERVER-85722 Investigate assumption that mongo lay... Open
related to SERVER-82686 Investigate if we need to explicitly ... Backlog
related to SERVER-85228 Disable auto compact temporarily Closed
is related to SERVER-85688 Set stable timestamp to end of each o... Open
is related to WT-12333 Provide mechanism to (re)set whether ... Backlog
is related to WT-12345 Return an error when setting a timest... In Code Review
Assigned Teams:
Replication
Backport Requested:
v7.3
Sprint: Repl 2024-01-22, Repl 2024-02-05, Repl 2024-02-19
Participants:

 Description   

This ticket comes after the discussion in WT-12032 where it was found that the server sets the oldest ts after the stable ts by using the "force" field of the WT set_timestamp API. This is dangerous for the following reasons:

  • A checkpoint happens when the ts are out of order (for instance because of background compaction), see WT-12032.
  • Eviction happens when the ts are out of order which could result in data loss as update chains will pruned based on the oldest timestamp.

Goals of this ticket:

  • Check if we can avoid setting the oldest timestamp greater than the stable timestamp in all cases.
  • Check if we can use "force" less liberally than we currently are.


 Comments   
Comment by Etienne Petrel [ 11/Jan/24 ]

During initial sync, when the stable timestamp is set to (0, 0), is that considered a valid stable timestamp by WT? It seems we assume that it is effectively a "null stable timestamp", is that assumption incorrect on our end?

0 is not a permitted value. Does the server call end up calling the WiredTiger API with a value set to 0?

It seems in WT-12032, it was mentioned that background compaction will internally trigger checkpoints. If those checkpoints are taken after initial sync has set the initial data timestamp, but before the first stable timestamp, what sort of checkpoints are they? Are they stable or unstable?

By default, background compaction triggers checkpoints with use_timestamp set to true:

    Config('use_timestamp', 'true', r'''
        if true (the default), create the checkpoint as of the last stable timestamp if timestamps
        are in use, or with all committed updates if there is no stable timestamp set. If false,
        always generate a checkpoint with all committed updates, ignoring any stable timestamp''',
        type='boolean'),

All committed updates are included in the checkpoints when no stable timestamp is set.

Generated at Thu Feb 08 06:55:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.