Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-13982

Copy across log files when initializing the live restore file system in file_size chunks.

    • Storage Engines
    • 5
    • StorEng - 2025-01-21

      In order to know which files need live restoring the live restore server walks the metadata file. WiredTiger has to complete recovery for the metadata file to be valid. So recovery must happen before the live restore background threads are created. This doesn't prevent the live restore file system from being created.

      If we consider that the log files are being read from a NAS or an S3FS the reads should be 1MB+ in size. However WiredTiger's log server does not support this use case. So to avoid doing lots of tiny reads to a network location we need to copy the log files to the destination directory prior to recovery. This is an optimization as technically it could read them from the source and still work fine.

      WiredTiger startup then has the following sequence:

      • Initialize the live restore FS and copy over the log files
      • Run recovery
      • Initialize the background threads

            Assignee:
            luke.pearson@mongodb.com Luke Pearson
            Reporter:
            luke.pearson@mongodb.com Luke Pearson
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: