Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-1745

Allow dirty eviction per file as checkpoint progresses

    • Type: Icon: Task Task
    • Resolution: Done
    • None
    • Affects Version/s: None
    • Component/s: None
    • None

      At the moment the transaction ID for a checkpoint causes us to hold all of the updates required to service that transaction in memory (i.e: blocks eviction). In a workload that consists of many short transactions and regular checkpoints those updates can be unnecessary.

      I'm planning to implement a scheme were:

      • We track a checkpoint generation.
      • Each file will reference it's current checkpoint generation. Once a checkpoint has finished in a particular file it will bump it's checkpoint generation.
      • We track the oldest transaction ID excluding the checkpoint ID, and the checkpoint ID.

      When checking whether a page can be evicted:

      • If checkpoint generation is equal to the current checkpoint generation: Use oldest ID.
      • If checkpoint generation is before the current checkpoint generation: Use older of oldest ID and checkpoint ID.

      Use a similar scheme in the obsolete update check of

      Unable to find source-code formatter for language: row_modify```. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml
      .
      
      All other places in the code use the oldest ID as a visibility check. To implement the visibility changes I'll likely split 

      __wt_txn_visible_all{{`}} into two variations - one as it currently exists and another that includes the checkpoint ID.

            Assignee:
            Unassigned Unassigned
            Reporter:
            alexander.gorrod@mongodb.com Alexander Gorrod
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: