Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-44410

Change backup cursor results to include a filesize

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.2.4, 4.3.4
    • Affects Version/s: None
    • Component/s: Storage
    • Labels:
      None
    • Fully Compatible
    • v4.2
    • Execution Team 2020-01-27, Execution Team 2020-02-10, Execution Team 2019-12-30

      Consider the case where the only changes to a WT file between incremental backups are to truncate its length.

      WT's API with duplicate cursors will neatly express this case to its caller (the server), but the current MongoDB API does not preserve that format going to its caller (network client). Without a change, a backup client could never safely shrink a file it was incrementally backing up.

      When WT returns the files to copy on the top-level backup cursor, it will also return a filelength that the backup application can safely truncate the file to. When opening up a duplicate backup cursor there may or may not be results. The proposed algorithm for MongoDB:

      1. MongoDB reads a <filename>, <filesize> pair from WT.
      2. MongoDB opens a "duplicate" cursor keyed by the <filename>.
      3. If the duplicate cursor returns EOF, MongoDB will add a document to the backup cursor stream with the following contents:
        { filename: <filename>,
          new_filesize: <filesize>,
          offset: 0,
          length: 0 }
        
      4. If the duplicate cursor returns specific blocks to copy, each with an <offset>, <length> pair, each document added to the backup cursor stream should be of the format:
        { filename: <filename>,
          new_filesize: <filesize>, // duplicated just like <filename>
          offset: <offset>,
          length: <length> }
        

            Assignee:
            gregory.wlodarek@mongodb.com Gregory Wlodarek
            Reporter:
            daniel.gottlieb@mongodb.com Daniel Gottlieb (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: