Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-69899

Dynamically size time series buckets based on workload and WT cache size

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 6.2.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Fully Compatible
    • Execution Team 2022-10-17, Execution Team 2022-10-31

      After SERVER-66690 and SERVER-66685, for high cardinality cases, we can run into WT cache pressure if we let buckets grow to the default size. In the case where we have a large number of active (open or archived) buckets and we allow the full default bucket size (125KiB), we could easily exceed the WT cache size and get into a state where each document write will trigger a write to disk.

      Instead, we should track the number of active buckets and lower the size limit dynamically so that all of the active buckets take up no more than half the WT cache. This will generally prevent us from getting into a constant write-through pattern, at the cost of slightly worse disk usage and query performance (due to smaller buckets). This offers the users at trade-off between instance size (RAM) and disk size.

            Assignee:
            fausto.leyva@mongodb.com Fausto Leyva (Inactive)
            Reporter:
            dan.larkin-york@mongodb.com Dan Larkin-York
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: