Make all instances of internalQuerySpillingMinAvailableDiskSpaceBytes.load() use a consistent internalQuerySpillingMinAvailableDiskSpaceBytes throughout query stages [Query Execution]

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Goal / Objective                                                                                          

      Ensure that all QE stages that read internalQuerySpillingMinAvailableDiskSpaceBytes see a consistent, snapshotted value for the full lifetime of a query. The value must be read once (at query initialization or stage construction) and held stable — not re-read from the underlying atomic on each spill or each stage — so that multiple spilling stages within the same query (e.g., two $sort stages, a $group  and a $lookup) all operate against the same threshold.

      Concretely, this means:
        - Pipeline stages with access to ExpressionContext must read the value through a static getter from via the QueryKnobCofiguration (the getter is currently not implemented)
        - SBE stages and sorter infrastructure that do not have ExpressionContext must snapshot the value at
        construction or open() time and store it as a member, rather than calling .loadRelaxed() at spill time.

      Context & Background

       internalQuerySpillingMinAvailableDiskSpaceBytes is an AtomicWord<long long> server parameter (default 500
        MB) that controls the minimum free disk space required before any spilling is permitted. Because it is an atomic, it can be read by calling .load()/.loadRelaxed() anywhere in the codebase. However, calling this way means we don't get a consistent internalQuerySpillingMinAvailableDiskSpaceBytes throughout a query.

      Call sites that should be considered to be updated

        The following query-execution-owned call sites currently call .load() directly on
        internalQuerySpillingMinAvailableDiskSpaceBytes and must be replaced with the appropriate snapshotted access
         pattern:

        File: src/mongo/db/exec/sort_executor.h
        Location: SortExecutor constructor
        ────────────────────────────────────────
        File: src/mongo/db/exec/sbe/stages/sort_stage_sort_impl.cpp
        Location: _makeSortOptions()
        ────────────────────────────────────────
        File: src/mongo/db/exec/sbe/stages/hashagg_base.cpp
        Location: HashAggBaseStage constructor
        ────────────────────────────────────────
        File: src/mongo/db/exec/sbe/stages/lookup_hash_table.cpp
        Location: LookupHashTable::open()
        ────────────────────────────────────────
        File: src/mongo/db/exec/sbe/stages/window.cpp
        Location: WindowStage constructor
        ────────────────────────────────────────
        File: src/mongo/db/exec/sbe/util/spilling.cpp
        Location: SpillingStore two-arg constructor
        ────────────────────────────────────────
        File: src/mongo/db/sorter/sorter_template_defs.h – I know this is under storex ownership, but I believe QE owns this spilling behavior.
        Location: InMemIterator::spill() (~line 193)
        ────────────────────────────────────────
        File: src/mongo/db/sorter/sorter_template_defs.h
        Location: InMemSorter::spill() (~line 738)
        ────────────────────────────────────────
        File: src/mongo/db/sorter/sorter_template_defs.h
        Location: BoundedSorter::_spill() (~line 1507)

      Acceptance Criteria

      All QE-owned stages have internalQuerySpillingMinAvailableDiskSpaceBytes see a consistent, snapshotted value for the full lifetime of a query, and to each sorter instance a QE stage constructs.

      Behavioral expectations
        - With default parameter settings, observable behavior of spilling is unchanged.
        - All stages within a single query use the same threshold value, regardless of when their spill operations occur relative to one another.

            Assignee:
            Unassigned
            Reporter:
            Stephanie Eristoff
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: