Positional array update behavior of applyOps on invalid field varies on different versions

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Works as Designed
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Replication
    • None
    • Repl 2019-03-25
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Positional array update of applyOps on an invalid field isn't consistent from 3.2 to 4.0.

      Here are the test and results from shane.harvey and john.morales.

      $ cat applyOps.js
// Run applyOps as mongomirror would.
// Use like: applyOps([{op: "i", ns:"test.test", o:{_id:1}}, {op: "i", ns:"test.test", o:{_id:2}}])
function applyOps(ops, extra) {
    // Force non-atomic applyOps mode for this batch
    ops.push({op: "c", ns: "admin.$cmd", o: {applyOps: [{op: "n", ns: "", o: {"msg": "noop"}}]}});
    var command = {
        applyOps: ops,
        writeConcern: {w: "majority"}
    };
    var res = db.adminCommand(command);
    // Avoid noise in output.
    delete res["operationTime"];
    delete res["$clusterTime"];
    return res
}

db.test.drop()
db.test.insert({_id:1, a: null});

// Test behavior in HELP-8472.
print('doc before: ' + tojson(db.test.findOne({_id: 1})));
var res = applyOps([{ns:"test.test", op:"u", o2:{_id:1}, o:{$set:{'a.0':2}}}]);
print('applyOps result: ' + tojson(res))

print('doc after : ' + tojson(db.test.findOne({_id: 1})));

      The behavior is different across releases.

      $ On 3.2 the server reports success and applies the update:
mongo applyOps.js
MongoDB shell version v4.0.1
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.2.18
WARNING: shell and server versions do not match
doc before: { "_id" : 1, "a" : null }
applyOps result: { "applied" : 2, "results" : [ true, true ], "ok" : 1 }
doc after : { "_id" : 1, "a" : { "0" : 2 } }

On 3.4 the server reports an error and does not apply the update:
$ mongo applyOps.js
MongoDB shell version v4.0.1
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.18
WARNING: shell and server versions do not match
doc before: { "_id" : 1, "a" : null }
applyOps result: {
    "applied" : 1,
    "code" : 16837,
    "codeName" : "Location16837",
    "errmsg" : "cannot use the part (a of a.0) to traverse the element ({a: null})",
    "results" : [
        false
    ],
    "ok" : 0
}
doc after : { "_id" : 1, "a" : null }

On 3.6, the server reverts back to the 3.2 behavior, reports success and applies the update:
$ mongo applyOps.js
MongoDB shell version v4.0.1
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.6.9
WARNING: shell and server versions do not match
doc before: { "_id" : 1, "a" : null }
applyOps result: { "applied" : 2, "results" : [ true, true ], "ok" : 1 }
doc after : { "_id" : 1, "a" : { "0" : 2 } }

Finally on 4.0, the server reports success and *does not apply the update*:
$ mongo applyOps.js
MongoDB shell version v4.0.1
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 4.0.4
doc before: { "_id" : 1, "a" : null }
applyOps result: { "applied" : 2, "results" : [ true, true ], "ok" : 1 }
doc after : { "_id" : 1, "a" : null }

      Because of the ambiguity, 0 is interpreted as a field name.

      However, there is no idempotency concern because if an error is returned, initial sync will refetch the document as if the document is missing; if the update succeeds, other fields are still updated correctly. Whatever intermediate state the field has, it will be cleaned up eventually since a later version doesn't have it. For example, if the field is an array eventually, there must be a $set to set the whole field to a valid array after the problematic update oplog entry.

      $ mongo --port 27018 applyOps.js 
MongoDB shell version v4.0.3
connecting to: mongodb://127.0.0.1:27018/
Implicit session: session { "id" : UUID("1f2b00ba-092d-4a1e-a633-8da75b61e436") }
MongoDB server version: 3.6.6
WARNING: shell and server versions do not match
doc before: { "_id" : 1, "a" : null }
applyOps result: { "applied" : 2, "results" : [ true, true ], "ok" : 1 }
doc after : { "_id" : 1, "a" : { "5" : 2 }, "b" : "b" }

      The bottom lines for the correctness of idempotency are:

      • Failed update reverts all changes on the document.
      • Successful update does partial update on other fields even if some fields are not expected.
      • Any structural and data type change of a field starts with an explicit $set, except setting to Object. For example, setting a non-existent field "a" to an array ["hello"] cannot be { $set: { "a.0": "hello" } }.

            Assignee:
            Siyuan Zhou
            Reporter:
            Siyuan Zhou
            Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved: