Dynamically tune WT_EVICT_MODIFY_COUNT_MIN based on workload characteristics

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Cache and Eviction
    • None
    • Storage Engines, Storage Engines - Transactions
    • None
    • 5

      The current implementation of WT_EVICT_MODIFY_COUNT_MIN (set to 15) is causing a regression in read-dominated workloads (95/5 read/write). This threshold was introduced to reduce aggressive eviction of dirty pages when the cache isn't near its upper threshold, allowing pages to accumulate more changes before reconciliation.

      However, this approach has a negative impact on read-dominated workloads where clean cache space is more valuable than keeping dirty pages in memory. 

      We want to implement a dynamic adjustment mechanism for WT_EVICT_MODIFY_COUNT_MIN based on workload characteristics. The system should:

      1. Monitor the number of updates reconciled per page during checkpoints
      2. Adjust the threshold based on observed update patterns by:
        • Lowering the threshold for read-dominated workloads to allow more aggressive eviction of dirty pages
        • Increasing the threshold for update-heavy workloads to reduce reconciliation overhead

      This is related to WT-14646, and BF-38809

              Assignee:
              [DO NOT USE] Backlog - Storage Engines Team
              Reporter:
              Alana Huang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: