Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-9641

Provide a compaction test operation

    • 5
    • 2024-03-19 - PacificOcean

      Summary
      Provide a method of obtaining information about the expected benefit of compact, such as a method of obtaining the decile "testing for compaction" output that begins a compact operation.

      Motivation
      Compact is not currently interruptible (WT-9636) and may not be successful. However, the "testing for compaction" phase is relatively fast even on large files. A means of running a compaction test could inform whether compaction should be pursued, for users with large data.

      Once this feature is developed, the Storage Execution team would need to ensure a MongoDB command that leverages this WT feature

      How likely is it that this use case or problem will occur?
      This is not a very likely use-case as many users can rely on initial sync. However, for the small minority of MongoDB users that find they need compact on a large data set, the benefit would be great (low blast radius, high impact)

      More though, building diagnostics around file space usage over time could be a step toward improving file space usage over time.

      If the problem does occur, what are the consequences and how severe are they?
      If a user is in a position where they need compact, they are normally looking at complex operational maneuvers as alternatives. A serious consequence could be running out of disk space on a large shard in a sharded cluster.

      Is this issue urgent?
      No

      Acceptance Criteria (Definition of Done)
      A user can determine via a "compactionTest" command if running compaction might reasonably be able to reclaim 10% of the file, without running compaction.

      Testing
      (What all testing needs to be done as part of this ticket? Unit? Functional? Performance?Testing at MongoDB side?)
      Correctness testing with respect to alignment with what compact will actually reclaim. I wouldn't think we need to be precise, but a "reasonable effort" seems nice.

      Documentation update
      Unsure, but probably not.

      [Optional] Suggested Solution
      If we like the output of the existing "test for compaction code", I think re-using it seems good

      Backport?
      I don't see a strong need to backport any improvement here.

            Assignee:
            sean.watt@mongodb.com Sean Watt
            Reporter:
            eric.sedor@mongodb.com Eric Sedor
            Votes:
            2 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: