Don't force full backups of the history store file for every incremental backup.

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Backup
    • Storage Engines
    • StorEng - Defined Pipeline, SE Persistence - 2025-06-20
    • None

      This ticket is spurred by live restore but not a live restore change. There exists logic in the incremental backup code that will trigger a full backup for the history store every time. Even though the history store is a regular b-tree which can be copied piecewise. I believe the logic predates durable history which is likely why it was never modified. 

      For most database the history store is small but for some it could be many gigabytes, this change would effectively reduce the backup cursor instrumentality for all users.

      It should be made, independently of the live restore changes and soaked for some time in testing. Given the logic of incremental backup is also independent of the history store I believe it to be a low risk change.

      Example diff: 

       -   if (F_ISSET(othercb->incr_src, WT_BLKINCR_FULL) ||
       -     WT_PREFIX_MATCH(cb->incr_file, "WiredTiger")) {
       +   if ((F_ISSET(othercb->incr_src, WT_BLKINCR_FULL) ||
       +     WT_PREFIX_MATCH(cb->incr_file, "WiredTiger")) && !WT_STRING_MATCH(cb->incr_file, WT_HS_FILE, strlen(WT_HS_FILE))) {

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Luke Pearson
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: