Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-7787

Don't read pages for checkpoint cleanup when the cache is in aggressive mode

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • WT10.0.1, 5.0.2, 4.4.9, 5.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • 3
    • Storage - Ra 2021-07-26
    • v5.0

      A checkpoint is also tasked with garbage collection. It has to clean up the older content from a file that is now obsolete. To do so, the checkpoint-cleanup process can effectively visit and load all of the internal pages in a file. Doing so when the cache has already reached an aggressive state can be counterproductive. Not only reading of the pages into the cache for the cleanup can be slower, but the checkpoint itself could hold eviction in the aggressive state longer. Eviction of dirty content from a file doesn't happen while the checkpoint is running on that file, reducing the chances of eviction going back to its normal state.

      When the eviction is set to aggressive, the checkpoint could skip loading the internal pages that are not already in cache. This will help a faster checkpoint, and a higher chance for cache to get back to normal.

      Here is a change for this issue as written by haribabu.kommi:

      diff --git a/src/btree/bt_sync.c b/src/btree/bt_sync.c
      index d6a02ca1a..9a1579030 100644
      --- a/src/btree/bt_sync.c
      +++ b/src/btree/bt_sync.c
      @@ -385,6 +385,12 @@ __sync_page_skip(WT_SESSION_IMPL *session, WT_REF *ref, void *context, bool *ski
           if (ref->state != WT_REF_DISK)
               return (0);
      
      +    /* Don't read any pages when the cache is operating in aggressive mode. */
      +    if (__wt_cache_aggressive(session)) {
      +        *skipp = true;
      +        return (0);
      +    }
      +
           /* Don't read pages into cache during startup or shutdown phase. */
           if (F_ISSET(S2C(session), WT_CONN_RECOVERING | WT_CONN_CLOSING_TIMESTAMP)) {
               *skipp = true;
      

        1. ckpt-cleanup-pages.png
          71 kB
          Sulabh Mahajan

            Assignee:
            alison.felizzi@mongodb.com Alison Felizzi (Inactive)
            Reporter:
            sulabh.mahajan@mongodb.com Sulabh Mahajan
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: