Uploaded image for project: 'MongoDB ETL Tools'
  1. MongoDB ETL Tools
  2. TOOLS-807

Error when using mongorestore to replay oplog of setting binData field

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 3.0.3, 3.1.5
    • Fix Version/s: 3.0.8, 3.2.0-rc3
    • Component/s: mongorestore
    • Labels:
      None
    • Environment:
      Debian 7 amd64
    • Sprint:
      Server Tools C (11/23/15)

      Description

      When using mongorestore with option --oplogReplay to replay oplogs, I found a strange error that mongorestore cannot handle binData field's set operation. You maybe meet the same error if you do this:

      1. insert a test data.

      db.testData.insert({_id: 10000, data: BinData(0, "CgxVfs93PiT/DrxMSvASFgoNMTAuMTYwLjIyMi4xMhDEJxgKIAA="), size: 10})
      

      2. update its binData field.

              db.testData.update({_id: 10000}, {$set: {data: BinData(0, "")}})
      

      3. update its other field

              db.testData.update({_id: 10000}, {$set: {size: 20}})
      

      4. check with oplog

              use local
       
              db.oplog.rs.find().sort({$natural: -1})
      

      you may see the following response:

              { "ts" : Timestamp(1435505075, 1), "h" : NumberLong("311789885375292809"), "v" : 2, "op" : "u", "ns" : "test.testData", "o2" : { "_id" : 10000 }, "o" : { "$set" : { "size" : 20 } } }
              { "ts" : Timestamp(1435504921, 1), "h" : NumberLong("8559347228340080808"), "v" : 2, "op" : "i", "ns" : "test.testData", "o" : { "_id" : 10000, "data" : BinData(0,"CgxVfs93PiT/DrxMSvASFgoNMTAuMTYwLjI  yMi4xMhDEJxgKIAA="), "size" : 10 } }
      

      5. dump these two oplog and replay it

      In bash shell:

              mongodump --port 27017 -d local -c oplog.rs --query '{"ts" : {$gte: Timestamp(1435504921, 1)}}' -o ./oplogD/
       
              mv ./oplogD/local/oplog.rs.bson ./oplogR/oplog.bson
       
              mongorestore --port 27017 --oplogReplay ./oplogR/
      

      *after this* you would find data not as expected. In my own, data changes to this.

              { "_id" : 10000, "data" : BinData(0,"c2V0ABMAAAABc2l6ZQAAAAAAAAA0QAAAADIyMi4xMhDEJxgKIAA="), "size" : 20 }
      

      The size field is really correct, but the data field is not correct.

      6. The most *strange* thing would be this, if you dump only *one* oplog and replay it, the data would be correct.

              mongodump --port 27017 -d local -c oplog.rs --query '{"ts" : Timestamp(1435504921, 1)}' -o ./tmpD/
       
              mv ./tmpD/local/oplog.rs.bson ./tmpR/oplog.bson
       
              mongorestore --port 27017 --oplogReplay ./tmpR/
      

      After oplog replayed, the 'data' field is quite correct.

              { "_id" : 10000, "data" : BinData(0,"CgxVfs93PiT/DrxMSvASFgoNMTAuMTYwLjIyMi4xMhDEJxgKIAA="), "size" : 10 }
      

      Link in stackoverflow: http://stackoverflow.com/questions/31101768/error-when-using-mongorestore-to-replay-oplog-with-bindata-field

        Attachments

          Activity

            People

            • Assignee:
              gabriel.russell Gabriel Russell
              Reporter:
              zhanweelee Jonathan
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: