[SERVER-54019] Session migration from moveChunk can lead to higher 'n' and 'nModified' for retryable updates by _id Created: 25/Jan/21  Updated: 01/Feb/24  Resolved: 21/Nov/23

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 8.0.0-rc0

Type: Bug Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Abdul Qadeer
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-54127 Retryable update by _id may execute m... Backlog
Assigned Teams:
Sharding NYC
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

python buildscripts/resmoke.py run --suite=sharding repro_retryable_update_by_id_nmodified.js

repro_retryable_update_by_id_nmodified.js

(function() {
"use strict";
 
load("jstests/sharding/libs/create_sharded_collection_util.js");
 
const st = new ShardingTest({mongos: 1, config: 1, shards: 2, rs: {nodes: 1}});
 
const db = st.s.getDB("test");
const collection = db.getCollection("mycoll");
CreateShardedCollectionUtil.shardCollectionWithChunks(collection, {x: 1}, [
    {min: {x: MinKey}, max: {x: 0}, shard: st.shard0.shardName},
    {min: {x: 0}, max: {x: 10}, shard: st.shard0.shardName},
    {min: {x: 10}, max: {x: 20}, shard: st.shard1.shardName},
    {min: {x: 20}, max: {x: MaxKey}, shard: st.shard1.shardName},
]);
 
assert.commandWorked(collection.insert({_id: 0, x: 5, counter: 0}));
 
const sessionCollection = st.s.startSession({causalConsistency: false, retryWrites: false})
                              .getDatabase(db.getName())
                              .getCollection(collection.getName());
 
// Updates by _id are broadcasted to all shards which own chunks for the collection. After the
// session information is migrated to shard1 from the moveChunk command, both shard0 and shard1
// will report {n: 1, nModified: 1} for stmtId=0.
const updateCmd = {
    updates: [
        {q: {_id: 0}, u: {$inc: {counter: 1}}},
        {q: {_id: 10000}, u: {$inc: {counter: 1}}},
    ],
    txnNumber: NumberLong(0),
};
 
const firstRes = sessionCollection.runCommand("update", updateCmd);
assert.eq({n: firstRes.n, nModified: firstRes.nModified}, {n: 1, nModified: 1});
 
assert.commandWorked(
    db.adminCommand({moveChunk: collection.getFullName(), find: {x: 5}, to: st.shard1.shardName}));
 
const secondRes = sessionCollection.runCommand("update", updateCmd);
assert.eq({n: secondRes.n, nModified: secondRes.nModified}, {n: 1, nModified: 1});
 
st.stop();
})();

Participants:

 Description   

Mongos combining the write results from multiple shards can lead to ambiguity due to how shards may report 'n' and 'nModified' for the same statements.

[js_test:repro_retryable_update_by_id_nmodified] 2021-01-24T15:46:37.631+0000 s20023| {"t":{"$date":"2021-01-24T15:46:37.631+00:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn6","msg":"Slow query","attr":{"type":"command","ns":"test.mycoll","appName":"MongoDB Shell","command":{"update":"mycoll","txnNumber":0,"lsid":{"id":{"$uuid":"5bb5fa37-09a3-4036-9442-83fa45776750"}},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1611503197,"i":122}},"signature":{"hash":{"$binary":{"base64":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","subType":"0"}},"keyId":0}},"$db":"test"},"nShards":2,"nMatched":2,"nModified":2,"numYields":0,"reslen":185,"remote":"127.0.0.1:53194","protocol":"op_msg","durationMillis":2}}
[js_test:repro_retryable_update_by_id_nmodified] 2021-01-24T15:46:37.632+0000 uncaught exception: Error: [{ "n" : 2, "nModified" : 2 }] != [{ "n" : 1, "nModified" : 1 }] are not equal :



 Comments   
Comment by Githook User [ 20/Nov/23 ]

Author:

{'name': 'Abdul Qadeer', 'email': 'abdul.qadeer@mongodb.com', 'username': 'zorro786'}

Message: SERVER-81594 SERVER-54019 Add tests
Branch: master
https://github.com/mongodb/mongo/commit/a1c223397a0f64e9eb5a730c2d502904d60c1c23

Comment by Abdul Qadeer [ 16/Nov/23 ]

This will be fixed by the virtue of returning early in PM-3190 for updateOne with _id without shard key as soon as any one shard returns

{'n' : 1, 'nModified' : 1}

.

Generated at Thu Feb 08 05:32:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.