-
Type:
Bug
-
Resolution: Duplicate
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Query Execution
-
ALL
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Problem
Query sort spill flushing can run for over a second without observing cancellation. This occurs in both classic and SBE file-based sorters and is separate from the unchecked std::sort comparison phase.
Reproduction
Validated on master 59767b3c795 on ARM, using real data without failpoints or injected delays.
- Insert 120,000 documents with a random sort key and a 2KB string payload.
- Set internalQueryFrameworkControl to forceClassicEngine or trySbeEngine and internalQueryMaxBlockingSortMemoryUsageBytes to 1GB.
- Run an aggregation with [{$sort: {r: 1}}], allowDiskUse:true, and cursor batchSize:1. The sort completes in memory before the first response.
- Call releaseMemory on the still-open cursor. This flushes already-sorted results, excluding the earlier std::sort call.
- Sample interrupt tracking at 1.0 with overdueInterruptCheckIntervalMillis:30 and inspect this releaseMemory command's slow log and sort spill counters.
| Engine | Server duration, two runs | Interrupt checks | Maximum interrupt overdue time | Spills |
|---|---|---|---|---|
| Classic | 1110 / 1106ms | 4 / 4 | 1079 / 1075ms | 1 each |
| SBE | 1174 / 1171ms | 4 / 4 | 1144 / 1140ms | 1 each |
The isolated flush includes serialization, checksum/compression, file writes, and subsequent buffer destruction. These measurements do not separately attribute time among those phases. Natural 16MB-budget spilling also produced ticket delinquency, but includes sorting and does not isolate the writer.
Relevant code
- src/mongo/db/sorter/file_based_spiller.h: FileBasedSpiller::_spill loops over all buffered records without interrupt checks; SortedFileWriter serializes, compresses, and writes chunks.
- src/mongo/db/sorter/sorter_template_defs.h: InMemIterator::spill flushes the remaining already-sorted records, then clears buffers.
- Both SortExecutor and the SBE sort implementation use FileBasedSpiller.
Expected outcome
Bound cancellation-check gaps during file-based sort flushing and add regression coverage that isolates flushing from sorting. Preserve safe cleanup and spill-state consistency on cancellation. Check intermediate spill-run consolidation for the same issue; it was identified in code but not separately reproduced here.
Local runnable repro: /home/ubuntu/spill-delinquency-repros/repro.py --only sort. Repro source to be attached.
Related investigation: SPM-4353. Comparator interrupt checks are tracked separately in SERVER-135754.
- is duplicated by
-
SERVER-135754 Update $sort’s custom comparator to check for interrupt every few thousand documents.
-
- Closed
-