Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-65285

Gracefully handle empty group-by key when spilling in HashAgg

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 5.3.2
    • Affects Version/s: None
    • Component/s: None
    • Fully Compatible
    • ALL
    • v5.3
    • Hide 
      const insertColl = db.getDB("test").foo;
for (let i = 0; i < 500; ++i) {
    assert.commandWorked(insertColl.insert({a: i, string: "test test test"}));
}

assert.commandWorked(db.adminCommand(
    {setParameter: 1, internalQuerySlotBasedExecutionHashAggApproxMemoryUseInBytesBeforeSpill: 1}));assert.commandWorked(db.adminCommand(
    {setParameter: 1, internalQuerySlotBasedExecutionHashLookupApproxMemoryUseInBytesBeforeSpill: 1}));

pipeline = [{$lookup: {from: readColl.getName(), localField: "a", foreignField: "a", as: "results"}}];
      let res =
    readColl
        .aggregate(
            pipeline,
            {allowDiskUse: true})
        .toArray();

      Invariant tripped:

      {"t":{"$date":"2022-04-05T22:20:32.074+00:00"},"s":"F", "c":"ASSERT", "id":23081, "ctx":"conn1","msg":"Invariant failure","attr":{"expr":"size > 0","msg":"key size must be greater than 0","file":"src/mongo/db/record_id.h","line":92}}`
      Show
      const insertColl = db.getDB("test").foo; for (let i = 0; i < 500; ++i) { assert.commandWorked(insertColl.insert({a: i, string: "test test test"})); } assert.commandWorked(db.adminCommand( {setParameter: 1, internalQuerySlotBasedExecutionHashAggApproxMemoryUseInBytesBeforeSpill: 1}));assert.commandWorked(db.adminCommand( {setParameter: 1, internalQuerySlotBasedExecutionHashLookupApproxMemoryUseInBytesBeforeSpill: 1})); pipeline = [{$lookup: {from: readColl.getName(), localField: "a", foreignField: "a", as: "results"}}]; let res = readColl .aggregate( pipeline, {allowDiskUse: true}) .toArray(); Invariant tripped: {"t":{"$date":"2022-04-05T22:20:32.074+00:00"},"s":"F", "c":"ASSERT", "id":23081, "ctx":"conn1","msg":"Invariant failure","attr":{"expr":"size > 0","msg":"key size must be greater than 0","file":"src/mongo/db/record_id.h","line":92}}`
    • QE 2022-04-18

      I noticed when spilling to disk in $lookup plans that use HashAgg we can get into a situation where we only have a single row in the HashAgg stage with an empty group by key. If this spills we can trip an invariant in the server.

      I fixed this when merging spilling in HashLookupStage and added a C++ for this exact scenario in the HashAgg unittests. 

      We should use this ticket to backport the fix applied in SERVER-62739 which is to check the _inKeyAccessor.size() == 0 and bail out of spilling. 

      https://github.com/10gen/mongo/pull/4099/files#diff-e1665f068faa10a7e7304141f4f42562dc0a2c1691e77d434ec24c106b3baa66R264

      This is okay because we will only have a single row in the hash table and there's no point in spilling to disk because we will have spill and re-load the value into memory when updating it via point the switch accessor to the value and running the bytecode.

            Assignee:
            eric.cox@mongodb.com Eric Cox (Inactive)
            Reporter:
            eric.cox@mongodb.com Eric Cox (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: