-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Query Execution
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Goal / Objective
Ensure that all QE stages that read internalQuerySpillingMinAvailableDiskSpaceBytes see a consistent, snapshotted value for the full lifetime of a query. The value must be read once (at query initialization or stage construction) and held stable — not re-read from the underlying atomic on each spill or each stage — so that multiple spilling stages within the same query (e.g., two $sort stages, a $group and a $lookup) all operate against the same threshold.
Concretely, this means:
- Pipeline stages with access to ExpressionContext must read the value through a static getter from via the QueryKnobCofiguration (the getter is currently not implemented)
- SBE stages and sorter infrastructure that do not have ExpressionContext must snapshot the value at
construction or open() time and store it as a member, rather than calling .loadRelaxed() at spill time.
Context & Background
internalQuerySpillingMinAvailableDiskSpaceBytes is an AtomicWord<long long> server parameter (default 500
MB) that controls the minimum free disk space required before any spilling is permitted. Because it is an atomic, it can be read by calling .load()/.loadRelaxed() anywhere in the codebase. However, calling this way means we don't get a consistent internalQuerySpillingMinAvailableDiskSpaceBytes throughout a query.
Call sites that should be considered to be updated
The following query-execution-owned call sites currently call .load() directly on
internalQuerySpillingMinAvailableDiskSpaceBytes and must be replaced with the appropriate snapshotted access
pattern:
File: src/mongo/db/exec/sort_executor.h
Location: SortExecutor constructor
────────────────────────────────────────
File: src/mongo/db/exec/sbe/stages/sort_stage_sort_impl.cpp
Location: _makeSortOptions()
────────────────────────────────────────
File: src/mongo/db/exec/sbe/stages/hashagg_base.cpp
Location: HashAggBaseStage constructor
────────────────────────────────────────
File: src/mongo/db/exec/sbe/stages/lookup_hash_table.cpp
Location: LookupHashTable::open()
────────────────────────────────────────
File: src/mongo/db/exec/sbe/stages/window.cpp
Location: WindowStage constructor
────────────────────────────────────────
File: src/mongo/db/exec/sbe/util/spilling.cpp
Location: SpillingStore two-arg constructor
────────────────────────────────────────
File: src/mongo/db/sorter/sorter_template_defs.h – I know this is under storex ownership, but I believe QE owns this spilling behavior.
Location: InMemIterator::spill() (~line 193)
────────────────────────────────────────
File: src/mongo/db/sorter/sorter_template_defs.h
Location: InMemSorter::spill() (~line 738)
────────────────────────────────────────
File: src/mongo/db/sorter/sorter_template_defs.h
Location: BoundedSorter::_spill() (~line 1507)
Acceptance Criteria
All QE-owned stages have internalQuerySpillingMinAvailableDiskSpaceBytes see a consistent, snapshotted value for the full lifetime of a query, and to each sorter instance a QE stage constructs.
Behavioral expectations
- With default parameter settings, observable behavior of spilling is unchanged.
- All stages within a single query use the same threshold value, regardless of when their spill operations occur relative to one another.
- is related to
-
SERVER-121786 Make all instances of internalQuerySpillingMinAvailableDiskSpaceBytes.load() use a consistent internalQuerySpillingMinAvailableDiskSpaceBytes throughout query stages [Query Integration]
-
- Needs Scheduling
-