• Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: DHandles
    • None
    • Storage Engines - Foundations
    • 67.622
    • None
    • None

      Opening a table on disaggregated storage currently takes an exclusive write lock on the schema because the open path sometimes has to modify the schema. This blocks other threads from opening different tables at the same time. Separating the rare schema modifications from the normal read-only open path will let threads take a shared read lock and open tables concurrently.

      On disaggregated and layered storage, the open path can lazily mutate the schema. It can materialize stable tables (_layered_create_missing_stable_tables calling _wt_schema_create in conn_layered.c), publish a schema epoch, and bump next_file_id. Because the same schema lock guards both the frequent open/reopen operations and this rare mutation, the disaggregated open path must take an exclusive write lock for the whole path.

      WT-18928 converts the schema lock to a read/write lock and lets the non-disaggregated open path take it in shared mode, so opens of different tables run concurrently. Because of the nested mutation, the disaggregated path must keep the write lock and cannot run concurrently.

      This ticket tracks separating the lazy schema mutation on the disaggregated open path from the read-side open. This will allow disaggregated reopen to take the shared lock while the rare mutation runs under its own exclusive lock. First, we need a disaggregated high-dhandle workload to measure whether the exclusive open path is a real bottleneck.

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Mikhail Sotnikov (EXT)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: