Checkpoint and stat logging reopen handles closed in parallel by sweep

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • WT12.0.0, 9.1.0-rc0
    • Affects Version/s: None
    • Component/s: Checkpoints
    • None
    • Storage Engines - Transactions
    • 169.211
    • SE Transactions - 2026-08-28
    • 3
    • v9.0, v8.3, v8.0, v7.0
    • Not Needed

      Problem

      _wt_conn_btree_apply (src/conn/conn_dhandle.c) walks the connection's dhandle list to apply a function (checkpoint's per-tree work, or stat logging) to every currently-open btree, filtering out closed/dead/metadata handles under a momentary handle-list read lock. That read lock is dropped before conn_btree_apply_internal calls wt_session_get_dhandle. If the sweep server (sweep_expire -> sweep_expire_one -> sweep_close_dhandle_locked -> wt_conn_dhandle_close, src/conn/conn_sweep.c) closes the handle in that window, wt_session_get_dhandle finds it not-open, takes the exclusive write lock, and does a full reopen: wt_evict_file_exclusive_on + wt_btree_open — including the unconditional, blocking _btree_preload (src/btree/bt_handle.c:149, no config/session-flag gate).

      Why skipping (not reopening) is safe

      _sweep_close_dhandle_locked only attempts to close a handle when !btree->modified — i.e. every dirty page is already flushed. wt_conn_dhandle_close additionally calls _wt_checkpoint_close to flush before actually closing. So a handle sweep has closed is, by construction, fully durable: there is nothing left for a following checkpoint to write for that tree. Reopening it to checkpoint it again is redundant work, not necessary work.

      This is also already the intended behavior for the common case: __wt_conn_btree_apply's own filter skips handles that were already closed before the walk reached them. The bug is only that a handle closing during the walk falls through into the expensive reopen path instead of being treated the same way.

      Proposed fix

      Add a new flag, e.g. WT_DHANDLE_SKIP_REOPEN, checked inside _wt_session_lock_dhandle/wt_session_get_dhandle (session_dhandle.c) after the per-handle lock is acquired — the same lock sweep's close path takes — so the open/closed check is atomic with respect to sweep, closing the race rather than narrowing it. Set the flag unconditionally at the wt_session_get_dhandle call site inside conn_btree_apply_internal (conn_dhandle.c), since every caller of wt_conn_btree_apply (unnamed checkpoint, stat logging) is enumerating already-open handles and has no business reopening one that just closed. Named checkpoints/drops go through the separate _wt_meta_apply_all path and are unaffected.

            Assignee:
            Haribabu Kommi
            Reporter:
            Haribabu Kommi
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: