Sessions are published in the session array before they are marked internal

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Engines - Transactions
    • 21.465
    • None
    • None

      Problem

      _open_session publishes a session with WT_RELEASE_WRITE_WITH_BARRIER(session_ret->active, 1) (session_api.c), and only after that call returns does _wt_open_internal_session set WT_SESSION_INTERNAL.

      _wt_session_array_walk takes no lock. It reads active with an acquire barrier and then decides whether to skip the session based on WT_SESSION_INTERNAL (session_helper.c). A session observed inside that window looks active but not internal, so a walk invoked with skip_internal = true does not skip it. The connection close walk (conn_close_session_callback, conn_api.c) will then close it, and session_clear memsets the structure, out from under the thread that is opening it. _conn_rollback_transaction_callback has the same exposure, rolling back a transaction on a session it does not own.

      This violates the contract the barrier's own comment states: "Release write to ensure structure fields are set before any other thread will consider the session", and the paired comment in the walk: "guarantees that by the time this thread sees active == 1 all other fields in the session have been initialized properly". WT_SESSION_INTERNAL is simply not among the fields set before the barrier.

      Window size

      For callers passing open_metadata = false the window is tens of instructions, so it is narrow but real.

      For callers passing open_metadata = true it is much wider: _wt_open_session runs _wt_metadata_cursor() between publication and the caller's flag set, which takes the schema lock and opens a cursor. That affects tiered-server, statlog-server, hs_access, hs-open, compatibility-reconfig, txn rollback_to_stable, close_ckpt and startup-cleanup-and-verify.

      __wti_thread_group_resize (thread_group.c) also opens internal sessions when a group grows, and eviction auto-tunes its thread group, so an eviction thread's session can be caught the same way. Eviction is destroyed well after the connection close session walk.

      Suggested fix

      Plumb the flag set into _wt_open_session / _open_session and apply it before the release write, so the internal flag is visible to any thread that can see the session as active.

      Note when implementing: the flags have to be set before the error info initialization in _open_session, not merely before the barrier. _wt_session_reset_last_error early-returns on a session without WT_SESSION_SAVE_ERRORS, so setting the flags after it leaves err_info.err_msg NULL and breaks get_last_error for external sessions. test_error_info* catches this; the layered suites do not.

      Origin

      Split out of WT-18303. That ticket is a SIGSEGV in the deferred checkpoint pickup server during connection close, which this race explains; WT-18303 is fixed on its own by stopping that server before the session array walk, and does not depend on this change.

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Chenhao Qu
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: