[SERVER-47939] Improve format of documents returned by $backupCursor aggregation command Created: 04/May/20  Updated: 06/Dec/22  Resolved: 17/Aug/20

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: James Heppenstall Assignee: Backlog - Storage Execution Team
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-46076 Stream incremental backup data Closed
Assigned Teams:
Storage Execution
Participants:

 Description   

When we open a backup cursor and return documents via the $backupCursor aggregation command, documents currently have the following formats depending on the type of backup.

  • Non-incremental (i.e. full) backups return one document per file with the format:

    {
        filename: String,
        fileSize: Number
    }

  • The first full backup used as the basis for future incremental backups returns one document per file with the format:

    {
        filename: String,
        fileSize: Number,
        offset: Number (=0),
        length: Number (=fileSize)
    }

    Note: In this case, offset=0 and length=fileSize always holds.

  • Incremental backups return one document per unchanged file with the format:

    {
        filename: String,
        fileSize: Number
    }

  • Incremental backups return multiple documents (one document per block, where each block has a default size of 16MB) per changed file with the format:

    {
        filename: String,
        fileSize: Number,
        offset: Number,
        length: Number

We should make this format more consistent by returning document with the following format regardless of the type of backup.

{
    filename: String,
    fileSize: Number,
    offset: Number,
    length: Number
}

In the case of non-incremental backups and the first full backup used as the basis for future incremental backupsoffset=0 and length=0 would clearly indicate the type of backup. Since this involves additive changes to the backup API, it should require few changes from downstream teams like Cloud Backup.


Generated at Thu Feb 08 05:15:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.