<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:59:05 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-41924] Rollback occured when higher-priority PRIMARY rejoined replica set after storage failure</title>
                <link>https://jira.mongodb.org/browse/SERVER-41924</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Hello!&lt;/p&gt;

&lt;p&gt;We&apos;re testing MongoDB 4.0.10 failover capabilities and one of our test imitates block storage failure as described further.&lt;/p&gt;

&lt;p&gt;Replica set nodes:&lt;br/&gt;
mongo-test-arb1 MONGOS&lt;br/&gt;
mongo-test-arb2 MONGOS&lt;br/&gt;
mongo-test-db1 ARBITER&lt;br/&gt;
mongo-test-db2 PRIMARY&lt;br/&gt;
mongo-test-db3 SECONDARY&lt;br/&gt;
mongo-test-db4 SECONDARY&lt;br/&gt;
mongo-test-db5 SECONDARY&lt;br/&gt;
All nodes have 1 vote and priority 1 &lt;b&gt;except PRIMARY which has priority 10.&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;Test process:&lt;br/&gt;
1. Run a script writing 10000 documents to the replica set using 1 thread &lt;b&gt;with w:majority&lt;/b&gt;. Writes go through one of mongos instances.&lt;br/&gt;
2. While script is running, run this command on mongo-test-db2 (PRIMARY):&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;echo 1 &amp;gt; /sys/block/sda/device/delete ; sleep 300 ; echo b &amp;gt; /proc/sysrq-trigger&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;This leads to primary failure and client (pymongo in our case) receives an exception:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Write results unavailable from mongo-test-db2:27018 :: caused by :: Connection closed by peer&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;3. Wait for the script finished. As soon as it received 1 exception and not designed to repeat failed writes, we must have 9999 documents written to replica set at this point, so check db.collection.count() to ensure.&lt;br/&gt;
4. Wait for primary restarted (300 seconds) and recheck db.collection.count(). For such configuration I usually get numbers about 9200-9300. It means that &lt;b&gt;server rollbacks about 700 documents&lt;/b&gt; and I can see them in rollback directory in /var/lib.&lt;/p&gt;

&lt;p&gt;I repeated the test several times and noted that &lt;b&gt;if I set equal priorities to all nodes the problem does not occur&lt;/b&gt;. When primary has priority 1 it rejoins as secondary and successfully replicates all 9999 documents.&lt;/p&gt;

&lt;p&gt;Can someone explain such behavior of replica set? Is it a bug?&lt;/p&gt;</description>
                <environment></environment>
        <key id="814934">SERVER-41924</key>
            <summary>Rollback occured when higher-priority PRIMARY rejoined replica set after storage failure</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="4">Incomplete</resolution>
                                        <assignee username="daniel.hatcher@mongodb.com">Danny Hatcher</assignee>
                                    <reporter username="aanodin@gmail.com">Alexander A</reporter>
                        <labels>
                    </labels>
                <created>Wed, 26 Jun 2019 07:37:33 +0000</created>
                <updated>Sat, 16 Oct 2021 00:19:27 +0000</updated>
                            <resolved>Wed, 4 Sep 2019 18:38:51 +0000</resolved>
                                    <version>4.0.10</version>
                                                    <component>Replication</component>
                                        <votes>1</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="2348902" author="daniel.hatcher" created="Mon, 29 Jul 2019 20:54:31 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=aanodin&quot; class=&quot;user-hover&quot; rel=&quot;aanodin&quot;&gt;aanodin&lt;/a&gt;, have you had a chance to review my previous comment?&lt;/p&gt;</comment>
                            <comment id="2313618" author="daniel.hatcher" created="Fri, 5 Jul 2019 16:34:04 +0000"  >&lt;p&gt;I&apos;ve taken a look at the log files but it&apos;s hard to piece together exactly what is happening. I see nodes other than db-2 marked as DOWN as well as config server issues. Does the problem reproduce if you remove the mongos/config servers from the cluster so that its simply a 5-member replica set? Regardless, could you please test again and provide the exact timestamps, logs, &quot;diagnostic.data&quot; folders, and script you are using to insert to the documents?&lt;/p&gt;</comment>
                            <comment id="2306460" author="aanodin@gmail.com" created="Mon, 1 Jul 2019 10:29:36 +0000"  >&lt;p&gt;Hello Daniel,&lt;/p&gt;

&lt;p&gt;I&apos;ve uploaded log files via the form you provided. Hope you can see them.&lt;/p&gt;

&lt;p&gt;An experiment took place at June 25.&lt;/p&gt;</comment>
                            <comment id="2305272" author="daniel.hatcher" created="Fri, 28 Jun 2019 19:28:37 +0000"  >&lt;p&gt;Hello,&lt;/p&gt;

&lt;p&gt;In order for us to investigate, please provide the full &lt;tt&gt;mongod&lt;/tt&gt; log files from every node in the replica set. You can use our &lt;a href=&quot;https://10gen-httpsupload.s3.amazonaws.com/upload_forms/a8ccd1db-ce85-4ef6-8c31-f425bda3ab93.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Secure Uploader&lt;/a&gt; which only MongoDB employees can access.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 28 Jun 2019 19:28:37 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 28 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 28 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>aanodin@gmail.com</customfieldvalue>
            <customfieldvalue>daniel.hatcher@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hv77xb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|huwbhj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hv6u6n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>