[COMPASS-6215] Investigate changes in PM-3092: Truncate Ranges on Internal Collections Created: 19/Oct/22  Updated: 03/Nov/23  Resolved: 19/Oct/22

Status: Closed
Project: Compass
Component/s: None
Affects Version/s: None
Fix Version/s: No version

Type: Investigation Priority: Major - P3
Reporter: Backlog - Core Eng Program Management Team Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Documentation Changes: Not Needed

 Description   
Original Downstream Change Summary

StorE may introduce a new oplog entry

Description of Linked Ticket

Epic Summary

Summary

Use WiredTiger truncation to delete data on change stream pre/post-images collections and Serverless change collections.

Motivation

MongoDB maintains internal collections that contain full or partial copies of user data. These collections support features such as pre/post-images for change streams and per-tenant change streams for Serverless. These new collections multiply the amount of data we must insert and delete on each node. Unfortunately, deleting old data is still costly and slow despite improvements made in PM-2227. As a result the tasks that purge expired data may not be able to delete data faster than new data is inserted, which risks clusters running out of disk space.

WiredTiger has a truncation API that efficiently deletes large ranges of data without having to read everything from disk. Use this new API on these internal collections to delete data.

Documentation

[Scope Document|]
[Technical Design Document|]
[Product Description|]



 Comments   
Comment by PM Bot [ 03/Nov/23 ]

Fix Version updated for upstream PM-3092:
7.2.0

Comment by PM Bot [ 27/Jul/23 ]

Fix Version updated for upstream PM-3092:
7.2 Targeted

Comment by PM Bot [ 18/Jul/23 ]

Fix Version updated for upstream PM-3092:
7.3 Targeted

Generated at Wed Feb 07 22:42:19 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.