[SERVER-37076] Aggregation exchange partitioning does not work correctly for dotted keys, results in invariant() failure Created: 11/Sep/18  Updated: 29/Oct/23  Resolved: 08/Oct/18

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: 4.1.2
Fix Version/s: 4.1.4

Type: Bug Priority: Critical - P2
Reporter: David Storch Assignee: Unassigned
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-37077 Exchange boundaries are not validated... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

(function() {
    "use strict";
 
    db.c.drop();
    assert.commandWorked(db.c.insert({a: {b: -1}}));
    assert.commandWorked(db.c.insert({a: {b: 1}}));
 
    let result = assert.commandWorked(db.runCommand({
        aggregate: "c",
        pipeline: [],
        exchange: {
            policy: "range",
            consumers: NumberInt(2),
            bufferSize: NumberInt(1024),
            key: {"a.b": 1},
            boundaries: [{"a.b": MinKey}, {"a.b": 0}, {"a.b": MaxKey}],
            consumerids: [NumberInt(0), NumberInt(1)]
        },
        cursor: {batchSize: 0}
    }));
    assert.eq(result.cursors.length, 2);
    assert.eq(result.cursors[0].ok, true);
    assert.eq(result.cursors[1].ok, true);
    let firstCursorId = result.cursors[0].cursor.id;
    let secondCursorId = result.cursors[1].cursor.id;
 
    let firstGetMore =
        assert.commandWorked(db.runCommand({getMore: firstCursorId, collection: "c"}));
    printjson(firstGetMore);
 
    let secondGetMore =
        assert.commandWorked(db.runCommand({getMore: secondCursorId, collection: "c"}));
    printjson(secondGetMore);
}());

Sprint: Query 2018-09-24, Query 2018-10-08, Query 2018-10-22
Participants:

 Description   

When using range-based partitioning, the Exchange class is responsible for extracting the value of the key on which we are partitioning. In practice, this will be the shard key. This key extraction is not implemented properly when the key pattern contains a dotted field:

https://github.com/mongodb/mongo/blob/f45a68e589850bbf43c9d656bbbffa0d6b1efc79/src/mongo/db/pipeline/document_source_exchange.cpp#L284

This can cause an invariant to be tripped when attempting to assign an input document to a particular exchange partition. See the repro steps for a detailed example.



 Comments   
Comment by Githook User [ 08/Oct/18 ]

Author:

{'name': 'Martin Neupauer', 'email': 'martin.neupauer@mongodb.com', 'username': 'MartinNeupauer'}

Message: SERVER-37076 Aggregation exchange partitioning does not work correctly for dotted keys.
Branch: master
https://github.com/mongodb/mongo/commit/93e66d955ec346db14d85e0d3d1685ceccf0d043

Comment by Kyle Suarez [ 12/Sep/18 ]

I'm assigning this to Martin and pulling it forward into the current sprint. Though it's not expected for $exchange to work properly, we definitely want to prioritize addressing code that causes the server to crash. Otherwise, it causes noise in the fuzzer test suites and prevents us from releasing the code as-is, even if for a beta release.

Generated at Thu Feb 08 04:44:54 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.