Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-30470

Update subsystem doesn't preserve field-ordering on re-application of oplog entries

XMLWordPrintableJSON

    • ALL
    • Hide

      Apply the following patch and run the sync_tail_test C++ unit test.

      diff --git a/src/mongo/db/repl/sync_tail_test.cpp b/src/mongo/db/repl/sync_tail_test.cpp
index 2498a0cd9b..809314fec1 100644
--- a/src/mongo/db/repl/sync_tail_test.cpp
+++ b/src/mongo/db/repl/sync_tail_test.cpp
@@ -1425,6 +1425,22 @@ TEST_F(IdempotencyTest, ResyncOnRenameCollection) {
     ASSERT_EQUALS(runOp(op), ErrorCodes::OplogOperationUnsupported);
 }
 
+TEST_F(IdempotencyTest, FieldOrderingOnUpdateIsPreserved) {
+    ASSERT_OK(
+        ReplicationCoordinator::get(_opCtx.get())->setFollowerMode(MemberState::RS_RECOVERING));
+    ASSERT_OK(runOp(createCollection()));
+
+    auto insertOp = insert(fromjson("{ a: 0, _id: 1 }"));
+    ASSERT_OK(runOp(insertOp));
+
+    auto updateOp1 = update(1, fromjson("{ $unset: { a: null } }”));
+    auto updateOp2 = update(1, fromjson("{ $set: { 'a.c': 1, b: null } }"));
+    
+    auto ops = {updateOp1, updateOp2};
+    testOpsAreIdempotent(ops);
+}
+
 }  // namespace
 }  // namespace repl
 }  // namespace mongo
      Show
      Apply the following patch and run the sync_tail_test C++ unit test. diff --git a/src/mongo/db/repl/sync_tail_test.cpp b/src/mongo/db/repl/sync_tail_test.cpp index 2498a0cd9b..809314fec1 100644 --- a/src/mongo/db/repl/sync_tail_test.cpp +++ b/src/mongo/db/repl/sync_tail_test.cpp @@ -1425,6 +1425,22 @@ TEST_F(IdempotencyTest, ResyncOnRenameCollection) { ASSERT_EQUALS(runOp(op), ErrorCodes::OplogOperationUnsupported); } +TEST_F(IdempotencyTest, FieldOrderingOnUpdateIsPreserved) { + ASSERT_OK( + ReplicationCoordinator::get(_opCtx.get())->setFollowerMode(MemberState::RS_RECOVERING)); + ASSERT_OK(runOp(createCollection())); + + auto insertOp = insert(fromjson("{ a: 0, _id: 1 }")); + ASSERT_OK(runOp(insertOp)); + + auto updateOp1 = update(1, fromjson("{ $unset: { a: null } }”)); + auto updateOp2 = update(1, fromjson("{ $set: { 'a.c': 1, b: null } }")); + + auto ops = {updateOp1, updateOp2}; + testOpsAreIdempotent(ops); +} + } // namespace } // namespace repl } // namespace mongo
    • Query 2020-10-05

      The Oplog Application Idempotency project found the following bug:

      Starting with a document { _id: 1, a: 0 };

Apply: o: { $unset: { a: null } }
==> { _id: 1 };
Apply: o: { $set: { 'a.c': 1, b: null } }
==> { _id: 1, a: { c: 1 }, b: null };

// Applying the updates again:
Apply: o: { $unset: { a: null } }
==> { _id: 1, b: null };
Apply: o: { $set: { 'a.c': 1, b: null } }
==> { _id: 1, b: null, a: { c: 1 } };
      Impact

      Initial sync, replication rollback, and replication recovery following an unclean shutdown may apply a sequence of operations twice and therefore depend on oplog application being idempotent. It is possible that after applying a sequence of operations twice for a document to contain the same field-value pairs, but for the fields to be order differently within the document. While MongoDB replication subsystem has never promised to preserve the order of fields across documents on the primary and secondary, not doing so is problematic for two reasons:

      1. The "dbhash" command takes the md5sum of the contents of each collection. Having a different field ordering between the primary and secondary will lead to a dbhash mismatch. There was a similar issue with the $rename update modifier that was addressed in SERVER-21647.
      2. Queries of the form {a: {x: 1, y: 2}} will not match the document {_id: 0, a: {y: 2, x: 1}} (see SERVER-5030), so it is possible for the query when run a primary to return the document and for the query when run on a secondary to not return the document.

            Assignee:
            ian.boros@mongodb.com Ian Boros
            Reporter:
            may.hoque@mongodb.com May Hoque
            Votes:
            0 Vote for this issue
            Watchers:
            20 Start watching this issue

              Created:
              Updated:
              Resolved: