ExportXMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.1.0-rc0, 9.1.0-rc1026
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution
    • Fully Compatible
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Problem

      Destroying a large in-memory SBE hash lookup table produces interrupt delinquency during query cleanup. This happens without spilling. Cancellation cannot avoid required destruction, so this time should be distinguished from unchecked interruptible execution while retaining visibility into time needed to stop a query.

      Reproduction

      Validated on master 59767b3c795 on ARM with real data, no failpoints or injected delays.

      1. Insert 60,000 foreign documents with unique join keys and a 2KB string payload; insert 100 local documents with matching keys.
      2. Enable SBE and allowDiskUse. Raise hash-join eligibility limits to allow this foreign collection and set internalQuerySlotBasedExecutionHashLookupApproxMemoryUseInBytesBeforeSpill to 1GB so the hash table remains in memory.
      3. Verify explain selects EQ_LOOKUP with strategy HashJoin.
      4. Run a localField/foreignField lookup with cursor batchSize:1 and leave the cursor open.
      5. Issue killCursors for that cursor, with interrupt sampling at 1.0 and overdueInterruptCheckIntervalMillis:30.

      Observed killCursors: 171ms server duration, 1 interrupt check, 141ms interrupt overdue time, and zero spills. Fully draining the same in-memory query also produces an approximately 170ms final-getMore ticket hold and 140ms interrupt overdue time. The initial aggregate/build command is clean.

      Relevant code

      src/mongo/db/exec/sbe/stages/lookup_hash_table.cpp: LookupHashTable::reset destroys _memoryHt and clears the buffered materialized values. HashLookupStage::close calls reset.

      Expected outcome

      • Distinguish required bulk teardown from interruptible query execution in interrupt-delinquency accounting.
      • Preserve accounting of unchecked execution preceding cleanup.
      • Do not suppress cancellation during active execution or throw from unsafe cleanup paths.
      • Preserve visibility into teardown latency, using a separate measurement if needed, and add regression coverage.

      This is a cleanup issue, not evidence that the hash lookup spill writer never checks interrupts. Forced flushes made 300,008 interrupt checks while writing 120,000 records; their long total runtime was not one uninterrupted ticket hold.

      Local runnable repro: /home/ubuntu/spill-delinquency-repros/repro.py --only lookup --documents 60000. Repro source to be attached.
      Related investigation: SPM-4353; group cleanup precedent: SERVER-136021.

            Assignee:
            Kyle Burgess
            Reporter:
            Kyle Burgess
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: