<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:02:58 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-3407] oplog is not idempotent for array operators, which could lead to silent data corruption (without journalling)</title>
                <link>https://jira.mongodb.org/browse/SERVER-3407</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I was looking at bug &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3217&quot; title=&quot;$push and $pushAll oplog is not idempotent&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3217&quot;&gt;&lt;del&gt;SERVER-3217&lt;/del&gt;&lt;/a&gt;, and upon seeing the structure of the oplog I became worried that the only &quot;guarantee&quot; for idempotence might be a mere check on the size of the array on $push/$pop. This is certainly not enough to ensure idempotence.&lt;/p&gt;

&lt;p&gt;See, for example, the following code:&lt;/p&gt;

&lt;p&gt;// initialize db&lt;br/&gt;
db.foo.save(&lt;/p&gt;
{ x: [&quot;a&quot;, &quot;b&quot;] }
&lt;p&gt;)&lt;/p&gt;

&lt;p&gt;// do some sequence of operations:&lt;br/&gt;
db.foo.update({}, {$pull: { x: &quot;c&quot; }})&lt;br/&gt;
db.foo.update({}, {$push: { x: &quot;c&quot; }})&lt;br/&gt;
db.foo.update({}, {$addToSet: { x: &quot;d&quot; }})&lt;br/&gt;
db.foo.update({}, {$pull: { x: &quot;b&quot; }})&lt;br/&gt;
db.foo.update({}, {$pop: { x: 1 }})&lt;/p&gt;

&lt;p&gt;// we now have &lt;/p&gt;
{ x: [&quot;a&quot;, &quot;c&quot;] }
&lt;p&gt; in the database&lt;/p&gt;


&lt;p&gt;Now, when replaying the last 5 operations from the oplog, the $push and the $pop would be ignored as their $size does not match, but all other 3 operations would still occur:&lt;/p&gt;

&lt;p&gt;db.foo.update({}, {$pull: { x: &quot;c&quot; }})&lt;br/&gt;
db.foo.update({}, {$addToSet: { x: &quot;d&quot; }})&lt;br/&gt;
db.foo.update({}, {$pull: { x: &quot;b&quot; }})&lt;/p&gt;

&lt;p&gt;The database now contains &lt;/p&gt;
{ x: [&quot;a&quot;, &quot;d&quot;] }
&lt;p&gt;, which is incorrect.&lt;/p&gt;


&lt;p&gt;So this is pretty dramatic. I really think you ought to be looking for a formal proof of the idempotence of the oplog system if you are to make any claims about durability or even consistency of the data stored in MongoDB.&lt;/p&gt;</description>
                <environment></environment>
        <key id="19368">SERVER-3407</key>
            <summary>oplog is not idempotent for array operators, which could lead to silent data corruption (without journalling)</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="alerner">Alberto Lerner</assignee>
                                    <reporter username="makely">makely</reporter>
                        <labels>
                    </labels>
                <created>Mon, 11 Jul 2011 17:38:31 +0000</created>
                <updated>Tue, 12 Jul 2016 00:19:50 +0000</updated>
                            <resolved>Mon, 15 Oct 2012 22:14:20 +0000</resolved>
                                    <version>1.8.2</version>
                                    <fixVersion>2.3.0</fixVersion>
                                    <component>Replication</component>
                                        <votes>1</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="175126" author="alerner" created="Mon, 15 Oct 2012 22:14:20 +0000"  >&lt;p&gt;This was addressed by &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-7186&quot; title=&quot;Batched oplog application increases frequency of idempotency violations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-7186&quot;&gt;&lt;del&gt;SERVER-7186&lt;/del&gt;&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-4781&quot; title=&quot;replica set initial sync failure when update cannot be applied to a future version of an object received via clone&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-4781&quot;&gt;&lt;del&gt;SERVER-4781&lt;/del&gt;&lt;/a&gt; fixes.&lt;/p&gt;</comment>
                            <comment id="42683" author="eliot" created="Sun, 17 Jul 2011 02:01:38 +0000"  >&lt;p&gt;In 1.8.2 --journal has proven very stable.&lt;/p&gt;</comment>
                            <comment id="41922" author="makely" created="Tue, 12 Jul 2011 15:43:58 +0000"  >&lt;p&gt;Great! But given that it isn&apos;t the default yet, are there any caveats or do you now consider --journal mature enough to use in production if write performance isn&apos;t a concern?&lt;/p&gt;</comment>
                            <comment id="41864" author="eliot" created="Tue, 12 Jul 2011 12:14:44 +0000"  >&lt;p&gt;If you run 1.8.2 with --journal, there is no need for the oplog to be idempotent and this issue will never be encountered.&lt;/p&gt;</comment>
                            <comment id="41851" author="makely" created="Tue, 12 Jul 2011 09:39:38 +0000"  >&lt;p&gt;Thanks for the quick reply.&lt;/p&gt;

&lt;p&gt;This is great news for us as we&apos;re in the process of choosing a database and it means MongoDB still remains an option. Just to be clear, are you referring to a future version or are you saying that running MongoDB 1.8.2 with --journal implies that items from the oplog will never be run twice and thus the bug will never be encountered?&lt;/p&gt;</comment>
                            <comment id="41813" author="eliot" created="Tue, 12 Jul 2011 04:46:05 +0000"  >&lt;p&gt;We&apos;re actually probably going to be moving away from forcing the oplog to be idempotent with journalling.&lt;br/&gt;
It adds overhead, and with journalling isn&apos;t needed.  &lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="38107">SERVER-5789</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="49965">SERVER-6975</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="18022">SERVER-3217</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="39902">SERVER-5961</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="51688">SERVER-7186</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 12 Jul 2011 04:46:05 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        11 years, 18 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            11 years, 18 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>alerner</customfieldvalue>
            <customfieldvalue>eliot</customfieldvalue>
            <customfieldvalue>makely</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrovy7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrggf3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9572</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlk53:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>