Dhandle scaling Perf: On standby, layered cursors can pick up new stable checkpoints with staggered delays

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Layered Tables
    • None

      This comes out of the investigation in WT-17617, where this idea was POC'd.  This ticket will work in concert with WT-18060.  In the data_handle workload, there will be some stable btrees updated in each checkpoint (even though WT-18060 reduces the number of btrees greatly).  On any btree, there may be many cursors open, and for a highly active btree, a large number of sessions may have cursors that are simultaneously reopened to the latest checkpoint, and requests are made on the btree (thundering herd).

      To quiet the herd, each layered cursors, on an individual basis, decides how much to randomly stagger its reopen.  Correctness is maintained because ingest tables continue to cover the "gap" between the two checkpoints.

      Part of the work here is also to measure before and after, and potentially tune (POC was tested with a maximum stagger of 50ms).

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Donald Anderson
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: