Unlink a dropped table's file outside the schema lock

XMLWordPrintableJSON

    • Storage Engines - Foundations
    • 355.776
    • SE Foundations - 2026-09-15
    • 5

      Problem:

      WT_SESSION::drop unlinks the data file inside __wt_meta_track_off -> __meta_track_apply while holding the schema and table write locks. The file handle is already closed at that point, so the unlink frees the file's extents; for a large file this takes seconds. HELP-97809 (700 GB database drop): ~25% of drop time in unlink under the schema lock, up to 580 threads queued on the lock, ~3.9 s waits on ordinary queries.

      Solution:

      • __drop_file chooses a rename target under the lock: <file>.<btree_id>.wtdrop, adding a .N suffix while the name exists. A rename must never replace an existing file, because replacing one frees its extents inside the rename.
      • __meta_track_apply renames the file to the target (O(1); frees the name so a same-name create after lock release cannot fail EEXIST in __wt_block_manager_create) and queues the unlink on a connection-level pending list. Tiered shared-object removal is queued the same way. If the rename fails, unlink in place and propagate the error as today.
      • __wt_drop_pending_apply drains the list immediately after the lock macros return: in __session_drop, the four internal __wt_schema_drop callers (hs_conn.c, conn_api.c chunk cache, meta_turtle.c partial backup, txn_recover.c), and __wti_connection_close. Drain failures are logged; the metadata is already durable.
      • Unlink in place (no rename, no queue): WT_CONN_IN_MEMORY, WT_CONN_LIVE_RESTORE_FS (live restore rename requires a fully migrated file), disagg file: entries without a local file, tiered local files.
      • Statistics: session_table_drop_deferred (pending), session_table_drop_deferred_applied.
      • The history store truncate stays under the schema lock.

      Semantics: file removal completes inside the drop call. With concurrent drops another session's drain may apply this drop's entry, so "file gone when drop returns" holds for single-threaded callers; the name is always freed under the lock. A crash between the metadata checkpoint and the unlink orphans <file>.<id>.wtdrop; cleanup is WT-18586.

      Out of scope: deferring work to a background thread and the history store truncate (WT-18586), disagg TRIM (WT-18386), the metadata checkpoint in __wt_meta_track_off.

            Assignee:
            Yury Ershov
            Reporter:
            Yury Ershov
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: