Dhandle scaling Perf: cooperative on demand background open of stable tables

XMLWordPrintableJSON

      On a follower checkpoint pickup, rather than having all new layered cursors open the new stable tables at the same time, we'd "spin off" an open in the background and continue to use the old stable table.

      Here's how it could work conceptually - we add two new flags on the dhandle: a ready flag and a wanted flag.  They only apply to stable tables at a checkpoint.  When about to open a new stable table, mark the dhandle as wanted, but don't wait - use the old stable table (which along with the ingest table, continues to satisfy correctness).

      There's new server (background) thread(s) that look for wanted flags, and open those dhandles, marking them ready.  Any layered open encountering a ready flag knows it is ready to open.  That's the cooperative part - the use of flags between layered open threads and server threads.

      This is conceptual to make it easy to understand - we may not need an explicit ready flag - there's a flag to indicate btree open that might be enough.

            Assignee:
            Donald Anderson
            Reporter:
            Donald Anderson
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: