Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-4072

Background compaction

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Backlog
    • Component/s: None
    • Labels:

      Description

      The WT_SESSION::compact command operates on a single table, but causes database-wide checkpoints. These repeat while compaction is making progress reducing the on-disk size of the table.

      While this process does not hold any lock that directly interrupts other operations, taking database-wide checkpoints is a heavyweight operation. If multiple tables require compaction, working through all of them can be inefficient.

      Instead, consider having a mode where WT_SESSION::compact marks a table as requiring compaction and returns immediately. The next time checkpoint visits the table, it can do some (preferably time-bounded) amount of compaction work. If no progress is made, the "requires compaction" flag is cleared and subsequent checkpoints will not attempt further compaction. If further compaction is required, the table should remain dirty so it is visited by subsequent checkpoints.

      Some design is required to (a) determine whether the "requires compaction" flag persists in the metadata; and (b) decide how much work to do during each checkpoint to avoid impacting the performance of concurrent operations.

        Attachments

          Activity

            People

            Assignee:
            backlog-server-storage-engines Backlog - Storage Engines Team
            Reporter:
            michael.cahill Michael Cahill
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated: