ExportXMLWordPrintableJSON

    • Query Execution
    • ALL
    • Hide

      Create a 9.0.4 single-shard cluster. Run the following mongosh script.

      // bulkWrite through mongos ignores collectionUUID on later nsInfo entries that
      // repeat a namespace. Run with: mongosh <uri> server_repro.js
      // Uses (and drops) the database "repro_bulkwrite_nsinfo_uuid".
      const d = db.getSiblingDB("repro_bulkwrite_nsinfo_uuid");
      const ns = d.getName() + ".c";
      const uuidOf = () => d.getCollectionInfos({name: "c"})[0].info.uuid;
      const check = (res) => {
        if (res.ok !== 1) throw new Error(JSON.stringify(res));
        return res;
      };
      
      // Create c, drop it, and create it again, so the name has a stale UUID and a
      // current one.
      function setup() {
        d.dropDatabase();
        check(d.createCollection("c"));
        const stale = uuidOf();
        if (!d.c.drop()) throw new Error("drop failed");
        check(d.createCollection("c"));
        return {stale, current: uuidOf()};
      }
      
      // Insert one document per nsInfo entry, op i targeting nsInfo entry i.
      function run(label, nsInfo, ids, expected) {
        const res = check(db.adminCommand({
          bulkWrite: 1,
          ordered: false,
          errorsOnly: false,
          nsInfo,
          ops: ids.map((_id, i) => ({insert: i, document: {_id}})),
        }));
        const items = res.cursor.firstBatch.map(
            r => r.ok ? "inserted" : `error ${r.code} (checked ${r.collectionUUID})`);
        print(`\n${label}`);
        items.forEach((item, i) => print(`  op ${i} (${ids[i]}): ${item}`));
        print(`  expected: ${expected}`);
        print(`  documents in ${ns}: ${JSON.stringify(d.c.find().toArray().map(doc => doc._id))}`);
      }
      
      print(`server ${db.version()}, ${db.hello().msg === "isdbgrid" ? "mongos" : "mongod"}`);
      
      let u = setup();
      print(`stale UUID ${u.stale}, current UUID ${u.current}`);
      run("1. stale, then current",
          [{ns, collectionUUID: u.stale}, {ns, collectionUUID: u.current}],
          ["stale", "current"],
          "op 0 fails with 361, op 1 inserted");
      
      u = setup();
      run("2. current, then stale",
          [{ns, collectionUUID: u.current}, {ns, collectionUUID: u.stale}],
          ["current", "stale"],
          "op 0 inserted, op 1 fails with 361");
      
      u = setup();
      run("3. no UUID, then stale",
          [{ns}, {ns, collectionUUID: u.stale}],
          ["none", "stale"],
          "op 0 inserted, op 1 fails with 361");
      
      u = setup();
      run("4. stale, then no UUID",
          [{ns, collectionUUID: u.stale}, {ns}],
          ["stale", "none"],
          "op 0 fails with 361, op 1 inserted");
      
      d.dropDatabase();
      

      Actual results (mongos 9.0.4)

      1. stale, then current
        op 0 (stale): error 361 (checked 79c80710-52fb-4d80-92b4-edd8d6114b35)
        op 1 (current): error 361 (checked 79c80710-52fb-4d80-92b4-edd8d6114b35)
        expected: op 0 fails with 361, op 1 inserted
        documents in repro_bulkwrite_nsinfo_uuid.c: []
      
      2. current, then stale
        op 0 (current): inserted
        op 1 (stale): inserted
        expected: op 0 inserted, op 1 fails with 361
        documents in repro_bulkwrite_nsinfo_uuid.c: ["current","stale"]
      
      3. no UUID, then stale
        op 0 (none): inserted
        op 1 (stale): inserted
        expected: op 0 inserted, op 1 fails with 361
        documents in repro_bulkwrite_nsinfo_uuid.c: ["none","stale"]
      
      4. stale, then no UUID
        op 0 (stale): error 361 (checked e1fb60c8-f0c1-43cf-a1b7-7d7499895948)
        op 1 (none): error 361 (checked e1fb60c8-f0c1-43cf-a1b7-7d7499895948)
        expected: op 0 fails with 361, op 1 inserted
        documents in repro_bulkwrite_nsinfo_uuid.c: []
      

      In case 1, the error for op 1 reports the stale UUID even though op 1's `nsInfo` entry carries the current one.

      Expected results

      Each op is checked against the collectionUUID of the nsInfo entry it references. The same script run directly against a 9.0.4 replica set produces exactly this:

      1. stale, then current   op 0: error 361   op 1: inserted   documents: ["current"]
      2. current, then stale   op 0: inserted    op 1: error 361  documents: ["current"]
      3. no UUID, then stale   op 0: inserted    op 1: error 361  documents: ["none"]
      4. stale, then no UUID   op 0: error 361   op 1: inserted   documents: ["none"]
      

      Where the entries are lost

      With "logComponentVerbosity: {command: 2}" on the shard primary, I captured the request mongos forwards for case 1. These captures came from an earlier version of the script that used the database "godriver3699_probe".

      9.0.4 mongos forwards one nsInfo entry carrying the stale UUID, and remaps both ops to it:

      nsInfo: [{ns: "godriver3699_probe.c",
                collectionUUID: UUID("ac880a1b-6d94-40ae-b21a-91842a598bcf"),
                shardVersion: {e: ObjectId("000000000000000000000000"), t: Timestamp(0, 0), v: Timestamp(0, 0)},
                databaseVersion: {...}}]
      ops:    [{insert: 0, document: {_id: "stale"}},
               {insert: 0, document: {_id: "current"}}]
      

      8.0.4 mongos forwards both entries unchanged:

      nsInfo: [{ns: "godriver3699_probe.c", collectionUUID: UUID("7427fd57-ae0c-494b-8304-235cfaaaab87")},
               {ns: "godriver3699_probe.c", collectionUUID: UUID("81615870-0803-4fd4-b2e0-26c6a160fe1d")}]
      ops:    [{insert: 0, document: {_id: "stale"}},
               {insert: 1, document: {_id: "current"}}]
      

      The shard then evaluates correctly the request it receives. The entries are lost in the router.

      Show
      Create a 9.0.4 single-shard cluster. Run the following mongosh script. // bulkWrite through mongos ignores collectionUUID on later nsInfo entries that // repeat a namespace. Run with : mongosh <uri> server_repro.js // Uses (and drops) the database "repro_bulkwrite_nsinfo_uuid" . const d = db.getSiblingDB( "repro_bulkwrite_nsinfo_uuid" ); const ns = d.getName() + ".c" ; const uuidOf = () => d.getCollectionInfos({name: "c" })[0].info.uuid; const check = (res) => { if (res.ok !== 1) throw new Error(JSON.stringify(res)); return res; }; // Create c, drop it, and create it again, so the name has a stale UUID and a // current one. function setup() { d.dropDatabase(); check(d.createCollection( "c" )); const stale = uuidOf(); if (!d.c.drop()) throw new Error( "drop failed" ); check(d.createCollection( "c" )); return {stale, current: uuidOf()}; } // Insert one document per nsInfo entry, op i targeting nsInfo entry i. function run(label, nsInfo, ids, expected) { const res = check(db.adminCommand({ bulkWrite: 1, ordered: false , errorsOnly: false , nsInfo, ops: ids.map((_id, i) => ({insert: i, document: {_id}})), })); const items = res.cursor.firstBatch.map( r => r.ok ? "inserted" : `error ${r.code} (checked ${r.collectionUUID})` ); print( `\n${label}` ); items.forEach((item, i) => print( ` op ${i} (${ids[i]}): ${item}` )); print( ` expected: ${expected}` ); print( ` documents in ${ns}: ${JSON.stringify(d.c.find().toArray().map(doc => doc._id))}` ); } print( `server ${db.version()}, ${db.hello().msg === "isdbgrid" ? "mongos" : "mongod" }` ); let u = setup(); print( `stale UUID ${u.stale}, current UUID ${u.current}` ); run( "1. stale, then current" , [{ns, collectionUUID: u.stale}, {ns, collectionUUID: u.current}], [ "stale" , "current" ], "op 0 fails with 361, op 1 inserted" ); u = setup(); run( "2. current, then stale" , [{ns, collectionUUID: u.current}, {ns, collectionUUID: u.stale}], [ "current" , "stale" ], "op 0 inserted, op 1 fails with 361" ); u = setup(); run( "3. no UUID, then stale" , [{ns}, {ns, collectionUUID: u.stale}], [ "none" , "stale" ], "op 0 inserted, op 1 fails with 361" ); u = setup(); run( "4. stale, then no UUID" , [{ns, collectionUUID: u.stale}, {ns}], [ "stale" , "none" ], "op 0 fails with 361, op 1 inserted" ); d.dropDatabase(); Actual results (mongos 9.0.4) 1. stale, then current op 0 (stale): error 361 (checked 79c80710-52fb-4d80-92b4-edd8d6114b35) op 1 (current): error 361 (checked 79c80710-52fb-4d80-92b4-edd8d6114b35) expected: op 0 fails with 361, op 1 inserted documents in repro_bulkwrite_nsinfo_uuid.c: [] 2. current, then stale op 0 (current): inserted op 1 (stale): inserted expected: op 0 inserted, op 1 fails with 361 documents in repro_bulkwrite_nsinfo_uuid.c: ["current","stale"] 3. no UUID, then stale op 0 (none): inserted op 1 (stale): inserted expected: op 0 inserted, op 1 fails with 361 documents in repro_bulkwrite_nsinfo_uuid.c: ["none","stale"] 4. stale, then no UUID op 0 (stale): error 361 (checked e1fb60c8-f0c1-43cf-a1b7-7d7499895948) op 1 (none): error 361 (checked e1fb60c8-f0c1-43cf-a1b7-7d7499895948) expected: op 0 fails with 361, op 1 inserted documents in repro_bulkwrite_nsinfo_uuid.c: [] In case 1, the error for op 1 reports the stale UUID even though op 1's `nsInfo` entry carries the current one. Expected results Each op is checked against the collectionUUID of the nsInfo entry it references. The same script run directly against a 9.0.4 replica set produces exactly this: 1. stale, then current op 0: error 361 op 1: inserted documents: ["current"] 2. current, then stale op 0: inserted op 1: error 361 documents: ["current"] 3. no UUID, then stale op 0: inserted op 1: error 361 documents: ["none"] 4. stale, then no UUID op 0: error 361 op 1: inserted documents: ["none"] Where the entries are lost With "logComponentVerbosity: {command: 2}" on the shard primary, I captured the request mongos forwards for case 1. These captures came from an earlier version of the script that used the database "godriver3699_probe". 9.0.4 mongos forwards one nsInfo entry carrying the stale UUID, and remaps both ops to it: nsInfo: [{ns: "godriver3699_probe.c", collectionUUID: UUID("ac880a1b-6d94-40ae-b21a-91842a598bcf"), shardVersion: {e: ObjectId("000000000000000000000000"), t: Timestamp(0, 0), v: Timestamp(0, 0)}, databaseVersion: {...}}] ops: [{insert: 0, document: {_id: "stale"}}, {insert: 0, document: {_id: "current"}}] 8.0.4 mongos forwards both entries unchanged: nsInfo: [{ns: "godriver3699_probe.c", collectionUUID: UUID("7427fd57-ae0c-494b-8304-235cfaaaab87")}, {ns: "godriver3699_probe.c", collectionUUID: UUID("81615870-0803-4fd4-b2e0-26c6a160fe1d")}] ops: [{insert: 0, document: {_id: "stale"}}, {insert: 1, document: {_id: "current"}}] The shard then evaluates correctly the request it receives. The entries are lost in the router.
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      When a bulkWrite command sent through a 9.0 mongos has two or more nsInfo entries with the same ns but different collectionUUID values, mongos merges them into one entry and forwards it to the shard. Only the first entry's collectionUUID is kept, and it is checked for every op that targets that namespace. A first entry with no collectionUUID counts too: all later UUIDs for that namespace are then dropped.

      As a result:

      • A write with a stale collectionUUID can succeed silently, inserting into the recreated collection. This happens whenever an earlier nsInfo entry for the same namespace carries the current UUID or no UUID.
      • Writes with a correct UUID, or with no UUID, can be rejected with "CollectionUUIDMismatch (361)" when an earlier entry for the same namespace carries a stale UUID.

      The same commands sent to a 9.0.4 replica set or 8.0.4 sharded cluster behave correctly.

            Assignee:
            [DO NOT USE] Backlog - Query Execution
            Reporter:
            Matt Dale
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: