-
Type:
Task
-
Resolution: Won't Do
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Live Restore
-
Storage Engines
-
SE Persistence - 2025-06-20
-
None
Incremental backup looks for previous IDs in the metadata file and in the logging system to know what point can be incremented from. This was design decision made some time ago and it means that running an incremental backup from a backup directory doesn't always work.
For example suppose we have 3 directories the original database taking backups (O), the backup directory that will be restored from (B), and the live restore directory which is the restore "destination" (R).
To take an incremental backup in O, we must first take a full backup with and ID# then subsequent incremental backups can be taken in directory O, this directory O is then in a valid state to be restored from.
Once we begin the restore in R it should not have knowledge of the incremental backup IDs in O because they may not exist in B. To get around this limitation we need to create a fake incremental backup id on commencement of live restore for the first time, so effectively creating a "start" id of 0, or 1.
This work is non-trivial as I have found that sometimes there may still be IDs in B which are then loaded. We need to:
- Understand when IDs go to B, I think the only case would be when they are stored in the metadata file
- Test various scenarios around this, both with logging enabled and disabled.
- Implement a similar change to the below change in live_restore_fs.c:
/* Allow incremental backups to be taken, during a live restore. This hacks around the fact we * won't have an incremental backup ID record in our history. */ WT_ASSERT(session, __wt_backup_set_blkincr(session, 0, 1048576, "ID1", 3) == 0);
- depends on
-
WT-14816 Conflate writes to the destination with data in the source file aligned with incremental backup granularity.
-
- Closed
-