Reduce the effect of the schema lock during WT_SESSION::create on high dhandle workloads

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: DHandles
    • None
    • Storage Engines, Storage Engines - Foundations
    • SE Foundations - Q3+ Backlog
    • None

      Running the default workload with a new dhandle benchmark from WT-15097, there is an interesting interaction between checkpoint, create and cursor_open API calls.  Checkpoint is fsync-ing files, and not holding the schema lock.  If cursor_opens are done with just a checkpoint, there's no contention, as cursor_open may need the schema lock for a short time to grab their dhandle.  However, inject creates into this mix.  Create holds the schema lock for the entirety of its operation, and (assuming create of a file: or simple table: ), it needs to create an initial (empty) btree file, and fsync the file before the metadata can be updated.  But, if there is a checkpoint happening, there might be a lot of fsync'ing happening, and this might increase the fsync time for a create dramatically.  Meanwhile, the schema lock is held, and the cursor opens must wait.

      With the time to do a cursor opens increasing by 20-30x as the workload grows to 30K+ tables, there is strong evidence that this interaction is a major cause of the slowdown.

              Assignee:
              [DO NOT USE] Backlog - Storage Engines Team
              Reporter:
              Donald Anderson
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: