-
Type: Task
-
Resolution: Fixed
-
Priority: Major - P3
-
Affects Version/s: None
-
Component/s: None
-
8
-
StorEng - 2023-02-07, 2023-02-23 "Stoney Baloney", 2023-03-21 Ellen Ripley, 2023-04-04 Bibbidi-Bobbidi-Boo
It would be useful for WiredTiger to validate/verify the data in a file along with the file's entries in the history store. The basic idea would be to have a function that is part of WT_SESSION::verify() and walks the contents of a file and its history together checking invariants. Ideally this can be done in such a way that it is trivial to add new variants.
Off the top of my head some possible invariants:
- If a key exists in the history store, it must also exist in the data file
- At most one value for a key can be visible at any timestamp.
Background
A form of this functionality was implemented during the durable history project. It was commented out because of WT-6676, which was fixed as part of WT-8447. This ticket was originally created to re-enable initial history verification code. But over time the initial history store verification bit-rotted as the code around it changed, and it was removed from the develop branch in WT-8572. That code might be a good starting point for this work. It can be found in the PR for WT-8572.
- depends on
-
WT-6637 Log recovery is nontimestamped and can overwrite some of the records in the checkpoint
- Closed
-
WT-6676 Fast truncate timestamp get lost after restart
- Closed
- is caused by
-
WT-6680 Temporarily disable history store verification
- Closed
- is depended on by
-
WT-10842 Improve the HS validation by checking hs_counter
- Open
-
WT-10844 Try to combine __wt_hs_verify_one and __hs_verify_id
- Open
-
WT-10841 Create a new connection configuration item to verify the HS
- Closed
- is related to
-
WT-5987 Turn off history store verification in the WT_SESSION.verify method
- Closed
-
WT-8496 Allow strict timestamp ordering for verify only when re-opening database
- Closed