ExportXMLWordPrintableJSON

    • Query Execution
    • Fully Compatible
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Analysis found unbounded check-free spans in the $group spill/merge paths:

      1. Classic merge loop — GroupProcessor::getNextSpilled() (src/mongo/db/pipeline/group_processor.cpp, ~line 53): the while loop merging all spilled partials for one group id runs unbounded within a single call (each iteration = disk read + deserialize + accumulator merge). With low-cardinality keys and a large spill this is seconds to minutes without a check. Add a direct, unamortized _expCtx->getOperationContext()->checkForInterrupt(); per iteration. Amortization does not work here: the whole output/merge phase makes fewer than 128 stage calls, so ExpressionContext::InterruptChecker never fires a real check during it, and each iteration can cost 100ms+ (merging large $push partials). Iteration counts are small (bounded by spill files per key), so the direct check is ~free. This also covers StreamingGroupStage and DocumentSourceGroup, which share GroupProcessor::getNext().
      2. Classic spill write loops — GroupProcessor::spill() (same file, ~lines 220-244): three switch branches each write up to a memory-limit's worth of groups via writer->addAlreadySorted(...) in one span, called per input doc during accumulation. Add _expCtx->checkForInterrupt(); per iteration in each of the three loops. The preceding in-memory std::sort stays unchecked (bounded by the memory limit; in-memory-sort bounding is covered by SERVER-135754).
      3. SBE merge loop — HashAggStage::getNextSpilled() (src/mongo/db/exec/sbe/stages/hash_agg.cpp, inner _rsCursor->next() loop ~lines 368-385): merges all spilled partials for one key within a single getNext(); the per-getNext checkForInterruptAndYield (~line 391) only bounds it per output key. Add a direct _opCtx->checkForInterrupt(); per iteration — unamortized for the same reason as item 1 (checkForInterruptNoYield would only check every 128 calls), and deliberately no yield to avoid save/restore mid-merge (sanctioned by the CanInterrupt comments in stages.h). BlockHashAggStage needs no change: its spill reads are bounded by kBlockOutSize per getNext().
      4. SBE spill write loop — HashAggBaseStage::spill() (src/mongo/db/exec/sbe/stages/hashagg_base.cpp, ~lines 168-171): writes the entire hash table to the spilling store via spillRowToDisk(...) in one check-free loop; the build phase feeding it is likewise unchecked. With 2M high-cardinality keys this produced a 671ms worst interrupt-check gap (1.9s total overdue in one operation). Add the SBE self-amortizing checkForInterruptNoYield(_opCtx); per row — amortized is fine here (high-volume cheap iterations, unlike items 1/3).
      5. In-memory teardown — GroupProcessorBase::reset() (src/mongo/db/pipeline/group_processor_base.cpp, ~line 119) destroys the whole in-memory GroupsMap in one check-free block — during dispose (via GroupBaseStage::doDispose()) and after each spill(). SBE has the same pattern: _ht->clear()/SimpleHashTable teardown in src/mongo/db/exec/sbe/stages/hashagg_base.cpp. Teardown cannot be usefully interrupted — it must complete regardless — so instead of adding checks, exclude it from interrupt delinquency by wrapping the whole teardown block in the existing OperationContext::runWithoutInterruptionExceptAtGlobalShutdown(): interrupt checks are not counted as delinquent while interrupts are ignored, and its scope guard resets the delinquency window on exit. The RAII scope covers everything in the block, including the spilling store (_recordStore) destruction, which validation showed is the dominant teardown block for spilled aggregations (3.6-4.3s for a 1M-key spill; a manual window reset placed before it re-attributed ~1.5s as delinquency). The preceding check-free execution span is accounted with checkForInterruptNoAssert() before entering the scope. Applied in GroupProcessorBase::reset(), HashAggStage::close(), and both _ht->clear() paths in HashAggBaseStage::spill().

      Empirical validation (standalone mongod at af69da80-base, overdueInterruptCheckSamplingRate=1.0, threshold 30ms, fast NVMe)

      • Classic spill write loop: 872ms worst gap, 2.7s total overdue in one 5s aggregation (2M keys, 5 spills).
      • Classic merge loop: 143-148ms worst gaps; zero interrupt checks during the entire output phase; the cheap-accumulator control stayed clean, so delinquency scales with per-iteration merge cost.
      • SBE spill write loop: 671ms worst gap, 1.9s total (2M keys).
      • Teardown: 114ms overdue (classic, 250k in-memory groups) and 31ms (SBE, 300k) on the final getMore — motivates the teardown item.

            Assignee:
            Kyle Burgess
            Reporter:
            Kyle Burgess
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: