Read shared metadata directly when opening the dhandle on a shared table on standby to avoid taking the checkpoint lock

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: DHandles, Metadata
    • None
    • Storage Engines - Foundations
    • None
    • None

      In WT-16292, we opted to take a checkpoint lock when trying to open the stable btree's dhandle because we need to ensure that we read the consistent checkpoint information for the history store and the data store. Otherwise, we may race with picking up a new checkpoint and see inconsistent checkpoints opened for the history store and the data store.

          /* Get the checkpoint information for this name/checkpoint pair. */
          if (checkpoint != NULL) {
              /*
               * Acquiring the checkpoint lock to prevent racing with picking up a new checkpoint.
               *
               * TODO: if we directly read from the shared metadata, we can avoid the lock here.
               */
              WT_WITH_CHECKPOINT_LOCK(session,
                ret = __btree_pin_hs_dhandle_and_get_meta_checkpoint(
                  session, btree, dhandle_name, checkpoint, &ckpt, &lr_fh_meta));
              WT_ERR(ret);
          }
      

      Taking this lock is expensive and may lead to stalls if it takes a long time to pick up a new checkpoint. For databases that have a lot of tables, it will become worse as we need to move more metadata from the shared metadata table to the local metadata table.

      To avoid this potential bottleneck, we can read the checkpoint information directly from the shared metadata table instead of the local metadata table to ensure we get the consistent checkpoint information for the history store and the data store.

      In this way, we can avoid taking the checkpoint lock.

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Chenhao Qu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: