<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:50:33 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-19311] mongoD becomes SECONDARY before finishing index builds during initial sync</title>
                <link>https://jira.mongodb.org/browse/SERVER-19311</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;When performing an initial sync on one of our sharded clusters, the replica transitions into SECONDARY mode before the background index builds are complete. This causes queries to run very slowly against this replica as the indexes are not complete.&lt;/p&gt;

&lt;p&gt;I have a log of this replica during the sync, but first wanted to ask how we should scrub before attaching here since it is our first time submitting a public ticket.&lt;/p&gt;</description>
                <environment></environment>
        <key id="215298">SERVER-19311</key>
            <summary>mongoD becomes SECONDARY before finishing index builds during initial sync</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="sam.kleinman">Sam Kleinman</assignee>
                                    <reporter username="dai@foursquare.com">Dai Shi</reporter>
                        <labels>
                    </labels>
                <created>Tue, 7 Jul 2015 17:58:19 +0000</created>
                <updated>Wed, 8 Jul 2015 13:43:42 +0000</updated>
                            <resolved>Wed, 8 Jul 2015 13:43:42 +0000</resolved>
                                    <version>2.6.10</version>
                                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="960176" author="samk" created="Tue, 7 Jul 2015 23:04:58 +0000"  >&lt;p&gt;It seems as if there are three approaches to this issue: &lt;/p&gt;

&lt;ol&gt;
	&lt;li&gt;Build all indexes in the foreground to prevent building indexes in the background during initial sync. If you build indexes before inserting data, the foreground index build may have a more limited impact on the availability of your system.&lt;/li&gt;
	&lt;li&gt;Ensure that only hidden and non-voting members of the system can perform routine initial syncs, either by reconfiguring the member before beginning the initial sync, or by removing the member before beginning the initial sync and then adding the new member as a hidden/non-voting member until the sync is complete.&lt;/li&gt;
	&lt;li&gt;When you&apos;re ready, you can consider upgrading to 3.0.x, which uses the new initial sync implementation and has the desired behavior. Transitioning to the WiredTiger storage engine may also provide related improvements in storage fragmentation.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;I&apos;m going to go ahead and close this issue, but feel free to open a support request and we can work on specific solutions.&lt;/p&gt;</comment>
                            <comment id="960026" author="dai@foursquare.com" created="Tue, 7 Jul 2015 20:47:48 +0000"  >&lt;p&gt;That is also not a great option since reconfig typically triggers a primary election, which causes many seconds of being unable to serve traffic. If we did this for all replica sets, we would serve a ton of site 500s both before and after the initial sync. This is not to mention the added work it would take to harden the code to automate reconfiguring replica sets and issuing the reconfig commands properly and ensuring they did the right thing each time (we have several hundred replica sets, so we can&apos;t do this by hand).&lt;/p&gt;</comment>
                            <comment id="960020" author="milkie" created="Tue, 7 Jul 2015 20:40:49 +0000"  >&lt;p&gt;You can make the member hidden before forcing an initial sync.&lt;/p&gt;</comment>
                            <comment id="960014" author="dai@foursquare.com" created="Tue, 7 Jul 2015 20:32:03 +0000"  >&lt;p&gt;This unfortunately is not an option for us, as we actually force an initial sync on one member of every replica set every day due to fragmentation. This only started happening after upgrading to 2.6, and was not an issue in 2.4.&lt;/p&gt;</comment>
                            <comment id="960011" author="milkie" created="Tue, 7 Jul 2015 20:28:13 +0000"  >&lt;p&gt;It is unlikely to be fixed in 2.4 or 2.6 because the behavior change was a side effect of rewriting the index building code.  In this situation, it is advisable to add a new node to a replica set as hidden and non-voting at first, to avoid reducing availability while the node is running initial sync.  Once the node has completed building all the indexes, reconfigure the set and promote the node to a full member.&lt;/p&gt;</comment>
                            <comment id="959960" author="dai@foursquare.com" created="Tue, 7 Jul 2015 19:35:37 +0000"  >&lt;p&gt;This seems like a pretty severe issue, since replicas that go into SECONDARY state before they have all the indexes queue like crazy in production. Are you saying the fix will not be backported to 2.6?&lt;/p&gt;</comment>
                            <comment id="959789" author="milkie" created="Tue, 7 Jul 2015 18:05:42 +0000"  >&lt;p&gt;I believe this behavior was corrected in MongoDB 3.0, whereupon initial sync will always build all indexes in the foreground.  It also builds them more efficiently than version 2.6.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 7 Jul 2015 18:05:42 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        8 years, 32 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>sam.kleinman</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            8 years, 32 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>dai@foursquare.com</customfieldvalue>
            <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>sam.kleinman</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrl16n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hsazt3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;Stop a secondary member in this replica set. Wipe all data in dbpath and restart. The replica will start initial sync, and somewhere in the middle of the index build stage the replica will transition to SECONDARY mode before the index build is finished.&lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsfvnz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>