<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:20:31 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-9474] Members not caught up to minvalid can later inadvertently delay state change to SECONDARY</title>
                <link>https://jira.mongodb.org/browse/SERVER-9474</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;If a member&lt;/p&gt;

&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;has not caught up to minvalid, and&lt;/li&gt;
	&lt;li&gt;has any non-trivial amount of additional replication lag&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;then it can inadvertently delay its transition to SECONDARY until it has fully eliminated its replication lag.  The expected behavior is for the member to transition to SECONDARY as soon as it catches up to minvalid.&lt;/p&gt;

&lt;p&gt;The bug is caused by the interaction of two pieces of functionality in &lt;tt&gt;SyncTail::oplogApplication()&lt;/tt&gt;: the batch size limit, and a timer that controls logic for attempting to transition to SECONDARY.  The timer is reset when a new batch is started, and then the timer generates a once-per-second call to &lt;tt&gt;tryToGoLiveAsASecondary()&lt;/tt&gt;; however, the first call happens only after the timer reaches t=1 second.  This can delay the &lt;tt&gt;tryToGoLiveAsASecondary()&lt;/tt&gt; call if batches take less than one second to process.&lt;/p&gt;

&lt;p&gt;This can be triggered in the following example scenario:&lt;/p&gt;

&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;A secondary is shut down while in the middle of processing an oplog batch (any time between the minvalid write and the oplog write)&lt;/li&gt;
	&lt;li&gt;The member is brought back into the replica set hours later&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;The member will call &lt;tt&gt;tryToGoLiveAsASecondary()&lt;/tt&gt; when it first starts up, but the SECONDARY transition will not occur because the minvalid condition will fail (as expected).  Then, it will start processing oplog entries.  Since it has replication lag, it will fetch oplog entries as fast as possible, and take less than one second each time to hit the batch limit.  It will thus never get to the next &lt;tt&gt;tryToGoLiveAsASecondary()&lt;/tt&gt; call until it has fully caught up.&lt;/p&gt;

&lt;p&gt;Note that if, in the above scenario, the secondary instead happened to have been shut down between batches, then it will transition to SECONDARY as soon as it is brought up (since the minvalid condition will succeed during the first &lt;tt&gt;tryToGoLiveAsASecondary()&lt;/tt&gt; call).&lt;/p&gt;</description>
                <environment></environment>
        <key id="73155">SERVER-9474</key>
            <summary>Members not caught up to minvalid can later inadvertently delay state change to SECONDARY</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="milkie@mongodb.com">Eric Milkie</assignee>
                                    <reporter username="rassi">J Rassi</reporter>
                        <labels>
                    </labels>
                <created>Thu, 25 Apr 2013 19:47:53 +0000</created>
                <updated>Mon, 11 Jul 2016 17:39:08 +0000</updated>
                            <resolved>Wed, 8 May 2013 17:59:07 +0000</resolved>
                                    <version>2.4.3</version>
                                    <fixVersion>2.5.0</fixVersion>
                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="332080" author="auto" created="Thu, 9 May 2013 16:53:51 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;date&apos;: u&apos;2013-05-09T15:03:52Z&apos;, u&apos;name&apos;: u&apos;Eric Milkie&apos;, u&apos;email&apos;: u&apos;milkie@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9474&quot; title=&quot;Members not caught up to minvalid can later inadvertently delay state change to SECONDARY&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9474&quot;&gt;&lt;del&gt;SERVER-9474&lt;/del&gt;&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9421&quot; title=&quot;queryoptimizer3.js failing during parallel tests basic and basicPlus&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9421&quot;&gt;&lt;del&gt;SERVER-9421&lt;/del&gt;&lt;/a&gt; fix replset tests with background index builds&lt;/p&gt;

&lt;p&gt;Lock the DBWrite lock before stopping index builds.&lt;br/&gt;
Avoid touching non-PODs while looking at non-threadsafe CurOp structures.&lt;br/&gt;
Fix the way we exit the batch-builder loop for replsettests&lt;br/&gt;
(this was broken by &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9474&quot; title=&quot;Members not caught up to minvalid can later inadvertently delay state change to SECONDARY&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9474&quot;&gt;&lt;del&gt;SERVER-9474&lt;/del&gt;&lt;/a&gt;&apos;s commit)&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/7008c37d48aa8893a4b79abd2ecca9699d8ee7d0&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/7008c37d48aa8893a4b79abd2ecca9699d8ee7d0&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="332078" author="auto" created="Thu, 9 May 2013 16:53:46 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;date&apos;: u&apos;2013-05-09T15:03:52Z&apos;, u&apos;name&apos;: u&apos;Eric Milkie&apos;, u&apos;email&apos;: u&apos;milkie@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9474&quot; title=&quot;Members not caught up to minvalid can later inadvertently delay state change to SECONDARY&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9474&quot;&gt;&lt;del&gt;SERVER-9474&lt;/del&gt;&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9421&quot; title=&quot;queryoptimizer3.js failing during parallel tests basic and basicPlus&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9421&quot;&gt;&lt;del&gt;SERVER-9421&lt;/del&gt;&lt;/a&gt; fix replset tests with background index builds&lt;/p&gt;

&lt;p&gt;Lock the DBWrite lock before stopping index builds.&lt;br/&gt;
Avoid touching non-PODs while looking at non-threadsafe CurOp structures.&lt;br/&gt;
Fix the way we exit the batch-builder loop for replsettests&lt;br/&gt;
(this was broken by &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9474&quot; title=&quot;Members not caught up to minvalid can later inadvertently delay state change to SECONDARY&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9474&quot;&gt;&lt;del&gt;SERVER-9474&lt;/del&gt;&lt;/a&gt;&apos;s commit)&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/7008c37d48aa8893a4b79abd2ecca9699d8ee7d0&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/7008c37d48aa8893a4b79abd2ecca9699d8ee7d0&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="331225" author="auto" created="Wed, 8 May 2013 17:20:30 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;date&apos;: u&apos;2013-05-08T16:59:51Z&apos;, u&apos;name&apos;: u&apos;Eric Milkie&apos;, u&apos;email&apos;: u&apos;milkie@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9474&quot; title=&quot;Members not caught up to minvalid can later inadvertently delay state change to SECONDARY&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9474&quot;&gt;&lt;del&gt;SERVER-9474&lt;/del&gt;&lt;/a&gt; always run through the entire batch-building loop at least once per batch&lt;/p&gt;

&lt;p&gt;Because we now run through the loop once with an empty op queue, we will always&lt;br/&gt;
do the stuff in &quot;occasionally check some things&quot; at least once per batch.  This&lt;br/&gt;
fixes &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9474&quot; title=&quot;Members not caught up to minvalid can later inadvertently delay state change to SECONDARY&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9474&quot;&gt;&lt;del&gt;SERVER-9474&lt;/del&gt;&lt;/a&gt;, where if you had a steady stream of writes, your node in&lt;br/&gt;
RECOVERING would never transition to SECONDARY until you caught up completely.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/25f2dfea762ebbaca65bd30d86429699b8a5da85&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/25f2dfea762ebbaca65bd30d86429699b8a5da85&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="331224" author="auto" created="Wed, 8 May 2013 17:20:28 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;date&apos;: u&apos;2013-05-08T16:59:51Z&apos;, u&apos;name&apos;: u&apos;Eric Milkie&apos;, u&apos;email&apos;: u&apos;milkie@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9474&quot; title=&quot;Members not caught up to minvalid can later inadvertently delay state change to SECONDARY&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9474&quot;&gt;&lt;del&gt;SERVER-9474&lt;/del&gt;&lt;/a&gt; always run through the entire batch-building loop at least once per batch&lt;/p&gt;

&lt;p&gt;Because we now run through the loop once with an empty op queue, we will always&lt;br/&gt;
do the stuff in &quot;occasionally check some things&quot; at least once per batch.  This&lt;br/&gt;
fixes &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9474&quot; title=&quot;Members not caught up to minvalid can later inadvertently delay state change to SECONDARY&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9474&quot;&gt;&lt;del&gt;SERVER-9474&lt;/del&gt;&lt;/a&gt;, where if you had a steady stream of writes, your node in&lt;br/&gt;
RECOVERING would never transition to SECONDARY until you caught up completely.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/25f2dfea762ebbaca65bd30d86429699b8a5da85&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/25f2dfea762ebbaca65bd30d86429699b8a5da85&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 8 May 2013 14:46:16 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        10 years, 40 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            10 years, 40 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>auto</customfieldvalue>
            <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>rassi</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrmvmf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrnhrj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>50837</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hszuyf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>