Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-1745

Allow dirty eviction per file as checkpoint progresses

    XMLWordPrintableJSON

Details

    • Task
    • Status: Closed
    • Resolution: Done
    • None
    • None
    • None
    • None

    Description

      At the moment the transaction ID for a checkpoint causes us to hold all of the updates required to service that transaction in memory (i.e: blocks eviction). In a workload that consists of many short transactions and regular checkpoints those updates can be unnecessary.

      I'm planning to implement a scheme were:

      • We track a checkpoint generation.
      • Each file will reference it's current checkpoint generation. Once a checkpoint has finished in a particular file it will bump it's checkpoint generation.
      • We track the oldest transaction ID excluding the checkpoint ID, and the checkpoint ID.

      When checking whether a page can be evicted:

      • If checkpoint generation is equal to the current checkpoint generation: Use oldest ID.
      • If checkpoint generation is before the current checkpoint generation: Use older of oldest ID and checkpoint ID.

      Use a similar scheme in the obsolete update check of

      .
       
      All other places in the code use the oldest ID as a visibility check. To implement the visibility changes I'll likely split 

      __wt_txn_visible_all{{`}} into two variations - one as it currently exists and another that includes the checkpoint ID.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              alexander.gorrod@mongodb.com Alexander Gorrod
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: