Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-1652

Size limited LSM tree

    • Type: Icon: Task Task
    • Resolution: Done
    • WT2.5.1
    • Affects Version/s: None
    • Component/s: None
    • Labels:

      There are workloads that want to have high insert throughput and a limit on the volume of the data stored. For example a workload that needs to store a certain amount of log files.

      WiredTiger already has an LSM implementation that supports high insert throughput. We could expose the LSM access method in a different way, so that we only keep a certain number of chunks in the tree (which would limit the volume of data).

      In short we would expose a way to set a limit on data size in an LSM tree (either a new file type, or a configuration setting to the existing LSM WT_SESSION::create group of options).

      When creating one of those tables we would:

      • Never do LSM background merges
      • When the total insert size (to the nearest LSM chunk size) exceeds the configured maximum add a new "drop oldest" work unit type, that drops the oldest chunk in the tree.

            Assignee:
            Unassigned Unassigned
            Reporter:
            alexander.gorrod@mongodb.com Alexander Gorrod
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: