Parallel page reconciliation during checkpoints

XMLWordPrintableJSON

    • Storage Engines - Persistence
    • 2,636.197
    • SE Persistence backlog
    • None

      Checkpoint currently reconciles all pages sequentially, which leads to potentially long and inefficient checkpoints, especially for single-table workloads (e.g., YCSB), and with continuous checkpointing (which we would like to use in SLS).

      One option for addressing this would be for the checkpoint code to add leaf pages from the btree traversal to a queue, and then have several threads to pick them up from the queue and then reconcile them in parallel. The checkpoint thread would then wait for these other threads to finish and reconcile the internal pages serially.

            Assignee:
            Peter Macko
            Reporter:
            Peter Macko
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: