Consider the case where the only changes to a WT file between incremental backups are to truncate its length.
WT's API with duplicate cursors will neatly express this case to its caller (the server), but the current MongoDB API does not preserve that format going to its caller (network client). Without a change, a backup client could never safely shrink a file it was incrementally backing up.
When WT returns the files to copy on the top-level backup cursor, it will also return a filelength that the backup application can safely truncate the file to. When opening up a duplicate backup cursor there may or may not be results. The proposed algorithm for MongoDB:
- MongoDB reads a <filename>, <filesize> pair from WT.
- MongoDB opens a "duplicate" cursor keyed by the <filename>.
- If the duplicate cursor returns EOF, MongoDB will add a document to the backup cursor stream with the following contents:
{ filename: <filename>, new_filesize: <filesize>, offset: 0, length: 0 }
- If the duplicate cursor returns specific blocks to copy, each with an <offset>, <length> pair, each document added to the backup cursor stream should be of the format:
{ filename: <filename>, new_filesize: <filesize>, // duplicated just like <filename> offset: <offset>, length: <length> }
- depends on
-
SERVER-45481 Change the backup API to return the blocks to copy for incremental backup
- Closed