[SERVER-3407] oplog is not idempotent for array operators, which could lead to silent data corruption (without journalling) Created: 11/Jul/11  Updated: 12/Jul/16  Resolved: 15/Oct/12

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: 1.8.2
Fix Version/s: 2.3.0

Type: Bug Priority: Major - P3
Reporter: makely Assignee: Alberto Lerner
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
is duplicated by SERVER-5789 $push is not idempotent Closed
is duplicated by SERVER-6975 $push idempotency issues with version... Closed
is duplicated by SERVER-3217 $push and $pushAll oplog is not idemp... Closed
Related
related to SERVER-5961 $push array size constraint can cause... Closed
is related to SERVER-7186 Batched oplog application increases f... Closed
Operating System: ALL
Participants:

 Description   

I was looking at bug SERVER-3217, and upon seeing the structure of the oplog I became worried that the only "guarantee" for idempotence might be a mere check on the size of the array on $push/$pop. This is certainly not enough to ensure idempotence.

See, for example, the following code:

// initialize db
db.foo.save(

{ x: ["a", "b"] }

)

// do some sequence of operations:
db.foo.update({}, {$pull: { x: "c" }})
db.foo.update({}, {$push: { x: "c" }})
db.foo.update({}, {$addToSet: { x: "d" }})
db.foo.update({}, {$pull: { x: "b" }})
db.foo.update({}, {$pop: { x: 1 }})

// we now have

{ x: ["a", "c"] }

in the database

Now, when replaying the last 5 operations from the oplog, the $push and the $pop would be ignored as their $size does not match, but all other 3 operations would still occur:

db.foo.update({}, {$pull: { x: "c" }})
db.foo.update({}, {$addToSet: { x: "d" }})
db.foo.update({}, {$pull: { x: "b" }})

The database now contains

{ x: ["a", "d"] }

, which is incorrect.

So this is pretty dramatic. I really think you ought to be looking for a formal proof of the idempotence of the oplog system if you are to make any claims about durability or even consistency of the data stored in MongoDB.



 Comments   
Comment by Alberto Lerner [ 15/Oct/12 ]

This was addressed by SERVER-7186 and SERVER-4781 fixes.

Comment by Eliot Horowitz (Inactive) [ 17/Jul/11 ]

In 1.8.2 --journal has proven very stable.

Comment by makely [ 12/Jul/11 ]

Great! But given that it isn't the default yet, are there any caveats or do you now consider --journal mature enough to use in production if write performance isn't a concern?

Comment by Eliot Horowitz (Inactive) [ 12/Jul/11 ]

If you run 1.8.2 with --journal, there is no need for the oplog to be idempotent and this issue will never be encountered.

Comment by makely [ 12/Jul/11 ]

Thanks for the quick reply.

This is great news for us as we're in the process of choosing a database and it means MongoDB still remains an option. Just to be clear, are you referring to a future version or are you saying that running MongoDB 1.8.2 with --journal implies that items from the oplog will never be run twice and thus the bug will never be encountered?

Comment by Eliot Horowitz (Inactive) [ 12/Jul/11 ]

We're actually probably going to be moving away from forcing the oplog to be idempotent with journalling.
It adds overhead, and with journalling isn't needed.

Generated at Thu Feb 08 03:02:58 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.