-
Type:
Technical Debt
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Reconciliation
-
None
-
Storage Engines, Storage Engines - Transactions
-
None
-
None
WiredTiger currently has a hardcoded limit on the maximum number of deltas: WT_DELTA_LIMIT.
The goal of this task is to remove that hard limit and update the code to dynamically handle any number of deltas.
The original goal of this work was to constrain things to a reasonable value during the initial disagg prototyping. The upsides of removing it are that we are forced to be able to handle arbitrarily long delta chains, and it prevents the fixed upper bound making its way into too many parts of the system.
There's also some experimentation value – it's possible there are workloads that would benefit from long delta chains, and removing the limit would make this experimentation possible.
Note that alongside WT_DELTA_LIMIT, this is enforced through the max_consecutive_deltas configuration option, but the macro isn't used in config since there's no nice way to share it.