[SERVER-72222] MapReduce with single reduce optimization fails when merging results in sharded cluster Created: 16/Dec/22  Updated: 29/Oct/23  Resolved: 20/Dec/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.4.19, 5.0.15, 6.0.4, 6.3.0-rc0

Type: Bug Priority: Major - P3
Reporter: Davis Haupt (Inactive) Assignee: Davis Haupt (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Problem/Incident
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.2, v6.1, v6.0, v5.0, v4.4
Steps To Reproduce:

Resmoke test:

(function() {
const st = new ShardingTest({
    shards: 2,
    mongos: 1,
    other: {
        mongosOptions: {setParameter: {mrEnableSingleReduceOptimization: true}},
        shardOptions: {setParameter: {mrEnableSingleReduceOptimization: true}},
    }
});
 
const mongosDB = st.s0.getDB(jsTestName());
const mongosColl = mongosDB[jsTestName()];
 
assert.commandWorked(mongosDB.dropDatabase());
assert.commandWorked(mongosDB.adminCommand({enableSharding: mongosDB.getName()}));
st.ensurePrimaryShard(mongosDB.getName(), st.shard0.shardName);
 
assert.commandWorked(
    mongosDB.adminCommand({shardCollection: mongosColl.getFullName(), key: {_id: 1}}));
 
// Split the collection into 4 chunks: [MinKey, -100), [-100, 0), [0, 100), [100, MaxKey).
assert.commandWorked(mongosDB.adminCommand({split: mongosColl.getFullName(), middle: {_id: 0}}));
 
// Move the [0, 100) and [100, MaxKey) chunks to shard1.
assert.commandWorked(mongosDB.adminCommand(
    {moveChunk: mongosColl.getFullName(), find: {_id: 50}, to: st.shard1.shardName}));
assert.commandWorked(mongosDB.adminCommand(
    {moveChunk: mongosColl.getFullName(), find: {_id: 150}, to: st.shard1.shardName}));
 
assert.commandWorked(mongosColl.insert({_id: -1}));
const map = function() {
    emit(0, {val: "mapped value"});
};
 
const reduce = function(key, values) {
    return {val: "reduced value"};
};
 
let res = assert.commandWorked(mongosDB.runCommand(
    {mapReduce: mongosColl.getName(), map: map, reduce: reduce, out: {inline: 1}}));
assert.eq(res.results[0], {_id: 0, value: {val: "mapped value"}});
 
assert.commandWorked(mongosColl.insert({_id: 1}));
 
res = assert.commandWorked(mongosDB.runCommand(
    {mapReduce: mongosColl.getName(), map: map, reduce: reduce, out: {inline: 1}}));
assert.eq(res.results[0], {_id: 0, value: {val: "reduced value"}});
 
st.stop();
}());

Sprint: QO 2022-12-26
Participants:
Case:

 Description   

This is because the output of $_accumulatorJsReduce has a different format depending on if `toBeMerged` is true or false. The codepath for the single reduce optimization added in SERVER-68766 does not change its output format accordingly. This ticket should make sure that the output of the accumulator respects the toBeMerged flag.



 Comments   
Comment by Githook User [ 21/Dec/22 ]

Author:

{'name': 'Davis Haupt', 'email': 'davis.haupt@mongodb.com', 'username': 'davish'}

Message: SERVER-72222 fix mapReduce single reduce optimization in sharded clusters
Branch: v4.4
https://github.com/mongodb/mongo/commit/ae230ddce14f29b72895e9e51925d87d1e7611d5

Comment by Githook User [ 21/Dec/22 ]

Author:

{'name': 'Davis Haupt', 'email': 'davis.haupt@mongodb.com', 'username': 'davish'}

Message: SERVER-72222 fix mapReduce single reduce optimization in sharded clusters
Branch: v5.0
https://github.com/mongodb/mongo/commit/897c960e0faf9ada0bc247567e57cb8b876324ee

Comment by Githook User [ 21/Dec/22 ]

Author:

{'name': 'Davis Haupt', 'email': 'davis.haupt@mongodb.com', 'username': 'davish'}

Message: SERVER-72222 fix mapReduce single reduce optimization in sharded clusters
Branch: v6.0
https://github.com/mongodb/mongo/commit/70bbc40c52b10395d0c54d63e4f7c35abadd8756

Comment by Githook User [ 19/Dec/22 ]

Author:

{'name': 'Davis Haupt', 'email': 'davis.haupt@mongodb.com', 'username': 'davish'}

Message: SERVER-72222 fix mapReduce single reduce optimization in sharded clusters
Branch: master
https://github.com/mongodb/mongo/commit/22897155fc8ead04abb9a9bc66ae2a00d44fbbc2

Generated at Thu Feb 08 06:21:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.