Uploaded image for project: 'Documentation'
  1. Documentation
  2. DOCS-9559

Docs for SERVER-10443: Compact command with LOW priority

    XMLWordPrintable

    Details

    • # Replies:
      1
    • Last comment by Customer:
      true

      Description

      Engineering Ticket Description:

      The 'compact' command should be runnable at low priority on any system without affecting existing performance.

      Current 'compact' command restrictions:

      • only runs replicas by default, unless forced;
      • locks entire collection, preventing any activity until finished.

      This case covers the trivial and straightforward case of regular maintenance. That is, the compact command should be runnable at any time, or set up as a background process. It should scan through all chunks, one at a time. The least recently used chunk should be addressed first.

      Processing:

      • The least recently used chunk should be mapped in and paged in to a special holding area, and actions performed on this chunk should be -
      • all documents sorted in primary/shard-key order;
      • documents snugged up against one another or spaced out, per requirements specified in compact command's padding factors;
      • OPTIONAL: As a further step, if any of this document's fields are indexed, those index entries should be verified and possibly corrected.
      • Processed chunk identifiers should be stored so the same chunks are not repeatedly processed in any n-hour period.

      This could be done without affecting performance by querying the locking percentage on the individual shards once per minute, and if the lock percentage is higher than 80%, pause compaction I/O. Alternatively, looking up IOSTAT's IO utilitization percentage will give an idea if the chunk's location can handle more IO.

      An additional parameter could be a rate limit on the number of chunks processed per minute/hour, or a max percentage of available IO to use.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              allison.moore Allison Reinheimer Moore
              Reporter:
              emily.hall Emily Hall
              Participants:
              Last commenter:
              Jonathan Dahl
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Days since reply:
                3 years, 39 weeks, 2 days ago
                Date of 1st Reply: