[SERVER-3407] oplog is not idempotent for array operators, which could lead to silent data corruption (without journalling) Created: 11/Jul/11 Updated: 12/Jul/16 Resolved: 15/Oct/12 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Replication |
| Affects Version/s: | 1.8.2 |
| Fix Version/s: | 2.3.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | makely | Assignee: | Alberto Lerner |
| Resolution: | Done | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||
| Description |
|
I was looking at bug See, for example, the following code: // initialize db ) // do some sequence of operations: // we now have { x: ["a", "c"] }in the database Now, when replaying the last 5 operations from the oplog, the $push and the $pop would be ignored as their $size does not match, but all other 3 operations would still occur: db.foo.update({}, {$pull: { x: "c" }}) The database now contains { x: ["a", "d"] }, which is incorrect. So this is pretty dramatic. I really think you ought to be looking for a formal proof of the idempotence of the oplog system if you are to make any claims about durability or even consistency of the data stored in MongoDB. |
| Comments |
| Comment by Alberto Lerner [ 15/Oct/12 ] |
|
This was addressed by |
| Comment by Eliot Horowitz (Inactive) [ 17/Jul/11 ] |
|
In 1.8.2 --journal has proven very stable. |
| Comment by makely [ 12/Jul/11 ] |
|
Great! But given that it isn't the default yet, are there any caveats or do you now consider --journal mature enough to use in production if write performance isn't a concern? |
| Comment by Eliot Horowitz (Inactive) [ 12/Jul/11 ] |
|
If you run 1.8.2 with --journal, there is no need for the oplog to be idempotent and this issue will never be encountered. |
| Comment by makely [ 12/Jul/11 ] |
|
Thanks for the quick reply. This is great news for us as we're in the process of choosing a database and it means MongoDB still remains an option. Just to be clear, are you referring to a future version or are you saying that running MongoDB 1.8.2 with --journal implies that items from the oplog will never be run twice and thus the bug will never be encountered? |
| Comment by Eliot Horowitz (Inactive) [ 12/Jul/11 ] |
|
We're actually probably going to be moving away from forcing the oplog to be idempotent with journalling. |