[SERVER-38897] Failure to update/delete orphan document fails multi write without shard key in transaction Created: 08/Jan/19  Updated: 17/Nov/23

Status: Backlog
Project: Core Server
Component/s: Sharding
Affects Version/s: 4.1.6
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Jack Mulrow Assignee: Backlog - Cluster Scalability
Resolution: Unresolved Votes: 0
Labels: ShardingRoughEdges, sharding-common-backlog
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Assigned Teams:
Cluster Scalability
Operating System: ALL
Steps To Reproduce:

(function() {
    "use strict";
 
    const dbName = "test";
    const collName = "foo";
    const ns = dbName + "." + collName;
 
    const st = new ShardingTest({shards: 2, config: 1});
 
    // Shard a collection with chunks [minKey, 0), [0, maxKey) on shard0 and shard1, respectively.
 
    assert.commandWorked(st.s.adminCommand({enableSharding: dbName}));
    st.ensurePrimaryShard(dbName, st.shard0.shardName);
 
    assert.commandWorked(st.s.adminCommand({shardCollection: ns, key: {skey: 1}}));
    assert.commandWorked(st.s.adminCommand({split: ns, middle: {skey: 0}}));
    assert.commandWorked(
        st.s.adminCommand({moveChunk: ns, find: {skey: 5}, to: st.shard1.shardName}));
 
    // Create a unique compound index prefixed on the shard key and insert two orphans with the same
    // shard key but different values for the second field in the index.
 
    const mongosColl = st.s.getDB(dbName)[collName];
    assert.commandWorked(mongosColl.createIndex({skey: 1, uniqueField: 1}, {unique: true}));
 
    assert.writeOK(mongosColl.insert({_id: 1, skey: -5, uniqueField: 0}));
 
    const orphanColl = st.rs1.getPrimary().getDB(dbName)[collName];
    assert.writeOK(orphanColl.insert({_id: 1, skey: -5, uniqueField: 0}));
    assert.writeOK(orphanColl.insert({_id: 2, skey: -5, uniqueField: 1}));
 
    // Start a transaction and send an update without the shard key that will be broadcast to all
    // shards. The write will fail against an orphan because it would violate the unique index,
    // which will abort the entire transaction. Outside a transaction, this write would succeed with
    // a writeError.
 
    const session = st.s.startSession();
    session.startTransaction();
    assert.commandFailedWithCode(session.getDatabase(dbName).runCommand({
        update: collName,
        updates: [{q: {uniqueField: 0}, u: {$set: {uniqueField: 1}}, multi: true}]
    }),
                                 ErrorCodes.DuplicateKey);
 
    // The non-orphan was not updated.
    assert.sameMembers(mongosColl.find().toArray(), [{_id: 1, skey: -5, uniqueField: 0}]);
 
    // Repeat the previous update outside a transaction and the command instead updates the
    // non-orphan with a writeError.
 
    assert.commandWorkedIgnoringWriteErrors(mongosColl.getDB().runCommand({
        update: collName,
        updates: [{q: {uniqueField: 0}, u: {$set: {uniqueField: 1}}, multi: true}]
    }));
 
    // The non-orphan was updated.
    assert.sameMembers(mongosColl.find().toArray(), [{_id: 1, skey: -5, uniqueField: 1}]);
 
    st.stop();
})();

Participants:

 Description   

Writes that target more than one shard (including multi=false writes with exact _id queries) are sent with shard versions in transactions, instead of being sent unversioned like they are outside of a transaction. Update and delete do not have a shard filter stage, so if an orphan document matches the write's query but cannot be updated or deleted (e.g. would violate a unique index), the entire operation will fail, whereas outside a transaction the response will contain a writeError, but the non-orphan documents will be successfully modified.



 Comments   
Comment by Mira Carey [ 13/Feb/20 ]

Attaching this to pm-1631 to investigate other ways we might start performing shard filtering for updates/deletes

Generated at Thu Feb 08 04:50:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.