Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-7871

Updates create $set operations in oplog in a new format that is not parseable by most drivers

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Duplicate
    • Affects Version/s: 2.2.1
    • Fix Version/s: None
    • Component/s: None
    • Environment:
      Same effect when server is running on Windows or CentOS.
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL

      Description

      The oplog starting in version 2.2.1 seemed to get a new format when updates are performed containing $set operations. This new format creates BSON documents in the rs.oplog collection where the $set key is included for each $set operation, whereas in versions prior to 2.2.1 there would only be a single $set key in the document and a subdocument would describe all the fields to be set. Since there are multiple $set keys, most drivers cannot parse these oplog documents, and will get exceptions while iterating through the oplog query results that contain any update operation with multiple $set updates.

      Here is an example for creating the malformed oplog entries on a new 2.2.1 or 2.2.2 system:

      rs0:PRIMARY> db.mycoll.save({_id:"unique-id-1234", "aaa":"0000", "bbb": "1111"})
      rs0:PRIMARY> db.mycoll.update({_id:"unique-id-1234"}, {$set:{"aaa":"1234","bbb":"7654"}})
      rs0:PRIMARY> use local
      rs0:PRIMARY> db.oplog.rs.find().pretty()
      {
              "ts" : Timestamp(1353099630000, 1),
              "h" : NumberLong(0),
              "v" : 2,
              "op" : "n",
              "ns" : "",
              "o" : {
                      "msg" : "initiating set"
              }
      }
      {
              "ts" : Timestamp(1353099716000, 1),
              "h" : NumberLong("6562103115049336849"),
              "v" : 2,
              "op" : "i",
              "ns" : "sample.mycoll",
              "o" : {
                      "_id" : "unique-id-1234",
                      "aaa" : "0000",
                      "bbb" : "1111"
              }
      }
      {
              "ts" : Timestamp(1353099904000, 1),
              "h" : NumberLong("-7902296395036453900"),
              "v" : 2,
              "op" : "u",
              "ns" : "sample.mycoll",
              "o2" : {
                      "_id" : "unique-id-1234"
              },
              "o" : {
                      "$set" : {
                              "aaa" : "1234"
                      },
                      "$set" : {
                              "aaa" : "1234"
                      }
              }
      }

      You will notice the "o" subdocument contains two "$set" keys, and drivers can't handle this new format. In prior versions, including 2.2.0, I would see this type of oplog entry when performing a update on a document, setting multiple fields:

      {
              "ts" : Timestamp(1353074108000, 3),
              "h" : NumberLong("-3950642131243057651"),
              "op" : "u",
              "ns" : "mydb.mycollection",
              "o2" : {
                      "_id" : "my-unique-id-1234"
              },
              "o" : {
                      "$set" : {
                              "aaa" : "1234",
                              "bbb" : "2345",
                              "ccc" : "3456",
                              "ddd" : "4567",
                              "eee" : "5678",
                              "fff" : "6789",
                              "ggg" : "7890",
                              "hhh" : "asdf"
                      }
              }
      }

      If it is helpful, here is the entire conversation in context.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: