ExportXMLWordPrintableJSON

    • Type: Bug
    • Resolution: Duplicate
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution
    • ALL
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Problem

      Query sort spill flushing can run for over a second without observing cancellation. This occurs in both classic and SBE file-based sorters and is separate from the unchecked std::sort comparison phase.

      Reproduction

      Validated on master 59767b3c795 on ARM, using real data without failpoints or injected delays.

      1. Insert 120,000 documents with a random sort key and a 2KB string payload.
      2. Set internalQueryFrameworkControl to forceClassicEngine or trySbeEngine and internalQueryMaxBlockingSortMemoryUsageBytes to 1GB.
      3. Run an aggregation with [{$sort: {r: 1}}], allowDiskUse:true, and cursor batchSize:1. The sort completes in memory before the first response.
      4. Call releaseMemory on the still-open cursor. This flushes already-sorted results, excluding the earlier std::sort call.
      5. Sample interrupt tracking at 1.0 with overdueInterruptCheckIntervalMillis:30 and inspect this releaseMemory command's slow log and sort spill counters.
      Engine Server duration, two runs Interrupt checks Maximum interrupt overdue time Spills
      Classic 1110 / 1106ms 4 / 4 1079 / 1075ms 1 each
      SBE 1174 / 1171ms 4 / 4 1144 / 1140ms 1 each

      The isolated flush includes serialization, checksum/compression, file writes, and subsequent buffer destruction. These measurements do not separately attribute time among those phases. Natural 16MB-budget spilling also produced ticket delinquency, but includes sorting and does not isolate the writer.

      Relevant code

      • src/mongo/db/sorter/file_based_spiller.h: FileBasedSpiller::_spill loops over all buffered records without interrupt checks; SortedFileWriter serializes, compresses, and writes chunks.
      • src/mongo/db/sorter/sorter_template_defs.h: InMemIterator::spill flushes the remaining already-sorted records, then clears buffers.
      • Both SortExecutor and the SBE sort implementation use FileBasedSpiller.

      Expected outcome

      Bound cancellation-check gaps during file-based sort flushing and add regression coverage that isolates flushing from sorting. Preserve safe cleanup and spill-state consistency on cancellation. Check intermediate spill-run consolidation for the same issue; it was identified in code but not separately reproduced here.

      Local runnable repro: /home/ubuntu/spill-delinquency-repros/repro.py --only sort. Repro source to be attached.
      Related investigation: SPM-4353. Comparator interrupt checks are tracked separately in SERVER-135754.

        1. repro-sort-flush.js
          2 kB
          Kyle Burgess

            Assignee:
            Unassigned
            Reporter:
            Kyle Burgess
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: