Investigate changes in SERVER-75439: Add metrics for preimage remover

XMLWordPrintableJSON

    • Type: Investigation
    • Resolution: Done
    • Priority: Major - P3
    • No version
    • Affects Version/s: None
    • Component/s: None
    • None
    • Not Needed

      Original Downstream Change Summary

      Added a new set of serverStatus metrics related to the pre-images remover/purging job.

      We've added a field changeStreamPreImages.purgingJob that contains the following attributes:

      • totalPass: Total number of deletion passes completed by the purging job.
      • docsDeleted: Cumulative number of pre-image documents deleted by the purging job
      • bytesDeleted: Cumulative size in bytes of all deleted documents from all pre-image collections by the purging job.
      • scannedCollections: Cumulative number of pre-image collections scanned by the purging job. In single-tenant environments this is the same as totalPass as there is 1 pre-image collection per tenant.
      • scannedInternalCollections: Cumulative number of internal pre-image collections scanned by the purging job. Internal collections are the segments of actual pre-images of collections within system.preimages.
      • maxStartWallTimeMillis: Maximum wall time from the first document of each pre-image collection.
      • timeElapsedMillis: Cumulative number of milliseconds elapsed since the first pass by the purging job.

        Description of Linked Ticket

        With the move to truncate markers we want to also keep track of the time spent performing truncation and the earliest wall time seen during truncation. This will help us detect possible issues with the system clock or clock shifts that prevent removal.

      The requirements for this are to record metrics similar in nature to change collection metrics:

      • Total time spent removing
      • Total passes done
      • Bytes deleted
      • Docs deleted
      • Collections scanned
      • Max wall clock seen in the preimage collection

      For multi-tenant environments this will be an aggregation between all the tenants so that one pass equals a scan on every tenant.

            Assignee:
            Unassigned
            Reporter:
            Backlog - Core Eng Program Management Team
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: