-
Type:
Story
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: History Store, Schema Management
-
Security Level: Public (Available to anyone on the web)
-
Storage Engines - Foundations, Storage Engines - Transactions
-
0
-
StorEng - Defined Pipeline
-
None
-
4
-
Disag_M13
For disaggregated storage we will support change streams using the same basic mechansims as attached storage. This means we need to support truncate operations on layered tables. In particular we need:
- Support for timestamped fast truncates (
WT-14778) - Layered table support for fast truncate on follower nodes (
WT-14679)
Currently changes streams work by reading information about past operations from the oplog. This is inefficient for a variety of reasons. For Disagg (and possibly 9.0 in general), the server would like to use durable history instead.
This would require the ability to have different history windows (i.e., oldest timestamps) for different tables. In practice, it may be sufficient to support two different history windows – a default (equivalent to the current 300 second window) and a larger window (24 hours?) for tables that have change streams enabled.