There are workloads that want to have high insert throughput and a limit on the volume of the data stored. For example a workload that needs to store a certain amount of log files.
WiredTiger already has an LSM implementation that supports high insert throughput. We could expose the LSM access method in a different way, so that we only keep a certain number of chunks in the tree (which would limit the volume of data).
In short we would expose a way to set a limit on data size in an LSM tree (either a new file type, or a configuration setting to the existing LSM WT_SESSION::create group of options).
When creating one of those tables we would:
- Never do LSM background merges
- When the total insert size (to the nearest LSM chunk size) exceeds the configured maximum add a new "drop oldest" work unit type, that drops the oldest chunk in the tree.