Explore compacting wt files via "punching a hole" in the file.

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Compaction
    • None
    • Storage Engines, Storage Engines - Persistence
    • None
    • None

      One way to reduce the disk usage of a WT file is to "punch holes" in the file when the filesystem supports it, so the underlying blocks are freed and returned to the filesystem.

      We could do this instead of physically moving blocks within the file.

      We don't need to limit this to the "compact" operation - we could do it after each checkpoint when we update the extent list for free blocks.

      A possible extension: always allocate new blocks at the end of the file and free blocks in the middle by punching holes.

      Potential side effects:

      • The reported file size may be much larger than the actual disk usage.
      • Copying these files with a regular cp may not be practical.
      • Customers may be confused about the real disk usage.
      • This mode could still be fine for professionals equipped with appropriate tools (e.g. Atlas).

              Assignee:
              [DO NOT USE] Backlog - Storage Engines Team
              Reporter:
              Yury Ershov
              Peter Macko, Sean Watt
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: