-
Type: Task
-
Resolution: Works as Designed
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Replication
-
Labels:None
-
Repl 2019-03-25
Positional array update of applyOps on an invalid field isn't consistent from 3.2 to 4.0.
Here are the test and results from shane.harvey and john.morales.
$ cat applyOps.js // Run applyOps as mongomirror would. // Use like: applyOps([{op: "i", ns:"test.test", o:{_id:1}}, {op: "i", ns:"test.test", o:{_id:2}}]) function applyOps(ops, extra) { // Force non-atomic applyOps mode for this batch ops.push({op: "c", ns: "admin.$cmd", o: {applyOps: [{op: "n", ns: "", o: {"msg": "noop"}}]}}); var command = { applyOps: ops, writeConcern: {w: "majority"} }; var res = db.adminCommand(command); // Avoid noise in output. delete res["operationTime"]; delete res["$clusterTime"]; return res } db.test.drop() db.test.insert({_id:1, a: null}); // Test behavior in HELP-8472. print('doc before: ' + tojson(db.test.findOne({_id: 1}))); var res = applyOps([{ns:"test.test", op:"u", o2:{_id:1}, o:{$set:{'a.0':2}}}]); print('applyOps result: ' + tojson(res)) print('doc after : ' + tojson(db.test.findOne({_id: 1})));
The behavior is different across releases.
$ On 3.2 the server reports success and applies the update: mongo applyOps.js MongoDB shell version v4.0.1 connecting to: mongodb://127.0.0.1:27017 MongoDB server version: 3.2.18 WARNING: shell and server versions do not match doc before: { "_id" : 1, "a" : null } applyOps result: { "applied" : 2, "results" : [ true, true ], "ok" : 1 } doc after : { "_id" : 1, "a" : { "0" : 2 } } On 3.4 the server reports an error and does not apply the update: $ mongo applyOps.js MongoDB shell version v4.0.1 connecting to: mongodb://127.0.0.1:27017 MongoDB server version: 3.4.18 WARNING: shell and server versions do not match doc before: { "_id" : 1, "a" : null } applyOps result: { "applied" : 1, "code" : 16837, "codeName" : "Location16837", "errmsg" : "cannot use the part (a of a.0) to traverse the element ({a: null})", "results" : [ false ], "ok" : 0 } doc after : { "_id" : 1, "a" : null } On 3.6, the server reverts back to the 3.2 behavior, reports success and applies the update: $ mongo applyOps.js MongoDB shell version v4.0.1 connecting to: mongodb://127.0.0.1:27017 MongoDB server version: 3.6.9 WARNING: shell and server versions do not match doc before: { "_id" : 1, "a" : null } applyOps result: { "applied" : 2, "results" : [ true, true ], "ok" : 1 } doc after : { "_id" : 1, "a" : { "0" : 2 } } Finally on 4.0, the server reports success and *does not apply the update*: $ mongo applyOps.js MongoDB shell version v4.0.1 connecting to: mongodb://127.0.0.1:27017 MongoDB server version: 4.0.4 doc before: { "_id" : 1, "a" : null } applyOps result: { "applied" : 2, "results" : [ true, true ], "ok" : 1 } doc after : { "_id" : 1, "a" : null }
Because of the ambiguity, 0 is interpreted as a field name.
However, there is no idempotency concern because if an error is returned, initial sync will refetch the document as if the document is missing; if the update succeeds, other fields are still updated correctly. Whatever intermediate state the field has, it will be cleaned up eventually since a later version doesn't have it. For example, if the field is an array eventually, there must be a $set to set the whole field to a valid array after the problematic update oplog entry.
$ mongo --port 27018 applyOps.js MongoDB shell version v4.0.3 connecting to: mongodb://127.0.0.1:27018/ Implicit session: session { "id" : UUID("1f2b00ba-092d-4a1e-a633-8da75b61e436") } MongoDB server version: 3.6.6 WARNING: shell and server versions do not match doc before: { "_id" : 1, "a" : null } applyOps result: { "applied" : 2, "results" : [ true, true ], "ok" : 1 } doc after : { "_id" : 1, "a" : { "5" : 2 }, "b" : "b" }
The bottom lines for the correctness of idempotency are:
- Failed update reverts all changes on the document.
- Successful update does partial update on other fields even if some fields are not expected.
- Any structural and data type change of a field starts with an explicit $set, except setting to Object. For example, setting a non-existent field "a" to an array ["hello"] cannot be { $set: { "a.0": "hello" } }.
- is duplicated by
-
SERVER-38747 Continually changing behavior of applyOps with an "invalid" update
- Closed
- related to
-
SERVER-38747 Continually changing behavior of applyOps with an "invalid" update
- Closed
-
SERVER-43043 Add idempotency test for two fields updates
- Closed