Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-38860

Positional array update behavior of applyOps on invalid field varies on different versions

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Works as Designed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Replication
    • Labels:
      None
    • Sprint:
      Repl 2019-03-25

      Description

      Positional array update of applyOps on an invalid field isn't consistent from 3.2 to 4.0.

      Here are the test and results from Shane Harvey and John Morales.

      $ cat applyOps.js
      // Run applyOps as mongomirror would.
      // Use like: applyOps([{op: "i", ns:"test.test", o:{_id:1}}, {op: "i", ns:"test.test", o:{_id:2}}])
      function applyOps(ops, extra) {
          // Force non-atomic applyOps mode for this batch
          ops.push({op: "c", ns: "admin.$cmd", o: {applyOps: [{op: "n", ns: "", o: {"msg": "noop"}}]}});
          var command = {
              applyOps: ops,
              writeConcern: {w: "majority"}
          };
          var res = db.adminCommand(command);
          // Avoid noise in output.
          delete res["operationTime"];
          delete res["$clusterTime"];
          return res
      }
       
      db.test.drop()
      db.test.insert({_id:1, a: null});
       
      // Test behavior in HELP-8472.
      print('doc before: ' + tojson(db.test.findOne({_id: 1})));
      var res = applyOps([{ns:"test.test", op:"u", o2:{_id:1}, o:{$set:{'a.0':2}}}]);
      print('applyOps result: ' + tojson(res))
       
      print('doc after : ' + tojson(db.test.findOne({_id: 1})));
      

      The behavior is different across releases.

      $ On 3.2 the server reports success and applies the update:
      mongo applyOps.js
      MongoDB shell version v4.0.1
      connecting to: mongodb://127.0.0.1:27017
      MongoDB server version: 3.2.18
      WARNING: shell and server versions do not match
      doc before: { "_id" : 1, "a" : null }
      applyOps result: { "applied" : 2, "results" : [ true, true ], "ok" : 1 }
      doc after : { "_id" : 1, "a" : { "0" : 2 } }
       
      On 3.4 the server reports an error and does not apply the update:
      $ mongo applyOps.js
      MongoDB shell version v4.0.1
      connecting to: mongodb://127.0.0.1:27017
      MongoDB server version: 3.4.18
      WARNING: shell and server versions do not match
      doc before: { "_id" : 1, "a" : null }
      applyOps result: {
          "applied" : 1,
          "code" : 16837,
          "codeName" : "Location16837",
          "errmsg" : "cannot use the part (a of a.0) to traverse the element ({a: null})",
          "results" : [
              false
          ],
          "ok" : 0
      }
      doc after : { "_id" : 1, "a" : null }
       
      On 3.6, the server reverts back to the 3.2 behavior, reports success and applies the update:
      $ mongo applyOps.js
      MongoDB shell version v4.0.1
      connecting to: mongodb://127.0.0.1:27017
      MongoDB server version: 3.6.9
      WARNING: shell and server versions do not match
      doc before: { "_id" : 1, "a" : null }
      applyOps result: { "applied" : 2, "results" : [ true, true ], "ok" : 1 }
      doc after : { "_id" : 1, "a" : { "0" : 2 } }
       
      Finally on 4.0, the server reports success and *does not apply the update*:
      $ mongo applyOps.js
      MongoDB shell version v4.0.1
      connecting to: mongodb://127.0.0.1:27017
      MongoDB server version: 4.0.4
      doc before: { "_id" : 1, "a" : null }
      applyOps result: { "applied" : 2, "results" : [ true, true ], "ok" : 1 }
      doc after : { "_id" : 1, "a" : null }
      

      Because of the ambiguity, 0 is interpreted as a field name.

      However, there is no idempotency concern because if an error is returned, initial sync will refetch the document as if the document is missing; if the update succeeds, other fields are still updated correctly. Whatever intermediate state the field has, it will be cleaned up eventually since a later version doesn't have it. For example, if the field is an array eventually, there must be a $set to set the whole field to a valid array after the problematic update oplog entry.

      $ mongo --port 27018 applyOps.js 
      MongoDB shell version v4.0.3
      connecting to: mongodb://127.0.0.1:27018/
      Implicit session: session { "id" : UUID("1f2b00ba-092d-4a1e-a633-8da75b61e436") }
      MongoDB server version: 3.6.6
      WARNING: shell and server versions do not match
      doc before: { "_id" : 1, "a" : null }
      applyOps result: { "applied" : 2, "results" : [ true, true ], "ok" : 1 }
      doc after : { "_id" : 1, "a" : { "5" : 2 }, "b" : "b" }
      

      The bottom lines for the correctness of idempotency are:

      • Failed update reverts all changes on the document.
      • Successful update does partial update on other fields even if some fields are not expected.
      • Any structural and data type change of a field starts with an explicit $set, except setting to Object. For example, setting a non-existent field "a" to an array ["hello"] cannot be { $set: { "a.0": "hello" } }.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                14 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: