Priority: Major - P3
Affects Version/s: None
It would be handy to throttle inserts if the cache becomes very dirty. Otherwise the amount of work required to complete a checkpoint can easily cause applications to stall waiting for space in the cache.
It's likely that we'll want to add throttling either when a page is first dirtied and/or when an updated is made to a dirty page.
This probably requires a new configuration setting to wiredtiger_open, something like eviction_dirty_max or eviction_dirty_throttle_trigger. We should throttle more and more aggressively as the proportion of dirty content moves above the setting (or approaches the setting if we want it to be a max).
We might want to allow the setting to be flexible when a checkpoint is running.
We will need new logic in the eviction server so that it starts choosing dirty pages to evict and becomes more aggressive about it as it reaches the target, independent of the current eviction_trigger.
The goals here are:
- Limit the amount of work required to complete a checkpoint
- Allow for very large caches to be helpful for read-mostly workloads
- Avoid application stalls due to checkpoints pinning transaction IDs and stopping eviction.