Investigate changes in PM-3092: Truncate Ranges on Internal Collections

XMLWordPrintableJSON

    • Type: Investigation
    • Resolution: Done
    • Priority: Major - P3
    • No version
    • Affects Version/s: None
    • Component/s: None
    • None
    • Not Needed

      Original Downstream Change Summary

      StorE may introduce a new oplog entry

      Description of Linked Ticket

      Epic Summary

      Summary

      Use WiredTiger truncation to delete data on change stream pre/post-images collections and Serverless change collections.

      Motivation

      MongoDB maintains internal collections that contain full or partial copies of user data. These collections support features such as pre/post-images for change streams and per-tenant change streams for Serverless. These new collections multiply the amount of data we must insert and delete on each node. Unfortunately, deleting old data is still costly and slow despite improvements made in PM-2227. As a result the tasks that purge expired data may not be able to delete data faster than new data is inserted, which risks clusters running out of disk space.

      WiredTiger has a truncation API that efficiently deletes large ranges of data without having to read everything from disk. Use this new API on these internal collections to delete data.

      Documentation

      [Scope Document|]
      [Technical Design Document|]
      [Product Description|]

            Assignee:
            Unassigned
            Reporter:
            Backlog - Core Eng Program Management Team
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: