Physical file unlink during drop runs under the schema lock

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Engines
    • 58.386
    • None
    • None

      __session_drop executes the whole of __wt_schema_drop under WT_WITH_SCHEMA_LOCK (plus the table write lock). The physical file removal is applied at the end of the drop by __wt_meta_track_off -> WT_ST_DROP_COMMIT -> __wt_block_manager_drop, i.e. still inside the schema lock (the dhandle-list write lock is released by then). Unlinking a large file can take seconds on common filesystems (extent freeing).

      Evidence from HELP-97809 (drop of a ~700 GB database, MongoDB 7.0.11): perf traces show ~25% of total drop time in unlink under the schema lock; FTDC shows up to 580 threads queued on the schema lock with ~3.9 s waits on ordinary queries (reopening swept dhandles requires the schema lock).

      Proposal (to be evaluated):

      • Move the physical file removal outside the schema lock - apply it after lock release or hand it to a background thread (sweep). If crash-consistency of the metadata-remove/unlink ordering matters, a rename-then-deferred-unlink scheme preserves it.

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Yury Ershov
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: