Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-6924

Cache usage higher than the configured cache size during checkpoints

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: In Code Review
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 5.0 Required
    • Component/s: None
    • Labels:
      None
    • Story Points:
      8
    • Sprint:
      Storage - Ra 2020-11-30, Storage - Ra 2020-12-14

      Description

      In diagnosing the root cause for WT-6681, we observed very high cache usage coincident with running checkpoints. In some instances, cache usage spiked to ~433% of the configured cache size. Our initial analysis shows that checkpointing non-history store (HS) pages can generate considerable HS content. As HS file only gets reconciled at the end of the checkpoint and there is no cache size check when inserting new HS contents, the cache usage can spike during checkpoint. Few points to be worked on for this ticket:

      1 - What is the role of flag WT_SESSION_IGNORE_CACHE_SIZE in this scenario?

      2 - A heuristic that prioritises HS pages for eviction was described in WT-6681 that helped bring down the cache usage down to ~135%. A valid question is why existing heuristics that were designed to prioritise eviction for cache dominating files didn't help?

      3 - We never fail checkpoint as of now. But how do we manage cases where checkpoint can not continue because cache is full?

      4 - Can we evict HS pages while checkpoint is running? If so, what are the restrictions (e.g., write gen)?

      5 -  Can we improve urgent eviction mechanism for this scenario?

        Attachments

        1. image-2020-11-18-13-58-44-858.png
          image-2020-11-18-13-58-44-858.png
          110 kB
        2. image-2020-11-23-11-40-14-538.png
          image-2020-11-23-11-40-14-538.png
          272 kB
        3. image-2020-11-23-17-26-16-721.png
          image-2020-11-23-17-26-16-721.png
          227 kB
        4. image-2020-11-24-12-49-26-839.png
          image-2020-11-24-12-49-26-839.png
          402 kB

          Issue Links

            Activity

              People

              Assignee:
              haseeb.bokhari Haseeb Bokhari
              Reporter:
              haseeb.bokhari Haseeb Bokhari
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated: