-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Storage Execution
-
Storage Execution 2026-10-26
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Problem
iteratorsMaxNum, the per-sorter cap on live spill-range iterators (and therefore cursors), is computed as iteratorsMaxBytesSize / sizeof(ContainerIterator): sorter_template_defs.h:546-559. For the container-based spiller, getIteratorSize() returns sizeof(ContainerIterator) (about 100-200 B): container_based_spiller.h:485-486. But each iterator holds a WiredTiger cursor whose real per-cursor cost is estimated at ~2 KB (kPerCursorOverheadBytes: container_based_spiller.h:462-463) plus a pinned leaf page.
So the cap is about 15-20x too high: the sorter permits thousands of cursors within a 1 MB "iterator budget". The same ~2 KB estimate is already used for the fan-in limit (getBufferSize(): container_based_spiller.h:493-500), so the two computations are inconsistent.
Proposed fix
Charge the real per-cursor cost (not sizeof) when computing iteratorsMaxNum for the container-based spiller, so the iterator budget reflects actual cursor memory. This depends on measuring the overhead accurately (SERVER-124382).
Notes
- Related: SERVER-136755 (per-index cursor pinning in the PDIB load phase), SERVER-124382 (per-cursor overhead measurement), SERVER-126498 (mergeSpills inefficiencies).
- depends on
-
SERVER-124382 Measure per-cursor memory overhead for kPerCursorOverheadBytes
-
- Backlog
-
- is related to
-
SERVER-136755 Primary-driven index builds pin all indexes' sorter cursors at once during the load phase
-
- Open
-
-
SERVER-124382 Measure per-cursor memory overhead for kPerCursorOverheadBytes
-
- Backlog
-
-
SERVER-126498 Improve inefficiencies in container-based sorter spiller mergeSpills
-
- Backlog
-