<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:53:34 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-20222] Index rebuilding on secondary during initial resync is not done in parallel for multiple collections or DBs</title>
                <link>https://jira.mongodb.org/browse/SERVER-20222</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Some setup info:&lt;br/&gt;
Mongo DB primary: 2.6.3&lt;br/&gt;
Mongo DB secondary: 3.0.5&lt;br/&gt;
Secondary uses wiredTiger&lt;/p&gt;

&lt;p&gt;When running an initial resync for an instance that contains 1000+ DBs, each with ~100 collections, with over 1000 indexes overall (per-db), the resync process is very slow. Specifically, the index creation process is very slow.&lt;br/&gt;
It seems that the index creation is performed one-by-one, without any parallelization between the collections or between DBs.&lt;/p&gt;

&lt;p&gt;It would be wise to rebuild many indexes in parallel, to fully utilize the available resources and shorten the resync process.&lt;/p&gt;</description>
                <environment></environment>
        <key id="227809">SERVER-20222</key>
            <summary>Index rebuilding on secondary during initial resync is not done in parallel for multiple collections or DBs</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="baruchoxman">Baruch Oxman</reporter>
                        <labels>
                    </labels>
                <created>Mon, 31 Aug 2015 18:58:47 +0000</created>
                <updated>Sat, 26 Sep 2015 12:22:41 +0000</updated>
                            <resolved>Sat, 26 Sep 2015 12:22:41 +0000</resolved>
                                    <version>3.0.5</version>
                                                    <component>Index Maintenance</component>
                    <component>Replication</component>
                    <component>WiredTiger</component>
                                        <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="1018997" author="baruchoxman" created="Tue, 1 Sep 2015 04:50:26 +0000"  >&lt;p&gt;Thanks Dan,&lt;br/&gt;
These other tickets seem to cover most of it.&lt;br/&gt;
The situation described in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-19766&quot; title=&quot;Initial sync index build is sequential and single-threaded&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-19766&quot;&gt;&lt;del&gt;SERVER-19766&lt;/del&gt;&lt;/a&gt; could actually be related - I&apos;m gonna check that.&lt;/p&gt;

&lt;p&gt;An additional improvement suggestions:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;Start rebuilding indexes on the secondary as soon as the secondary is done with syncing a DB&apos;s / collection&apos;s data, and in parallel with syncing of the next DB/collection - this would allow creating a &quot;pipeline&quot; mode, with a shorter overall delay, as in my usecase most DBs do not change when later the oplog is reapplied on the secondary.&lt;/li&gt;
	&lt;li&gt;Have a setting that would control the replication mode. For example, if I&apos;m willing to make a maintenance windown during which the primary is available only for the means of replication, aborting all writes to it, allow some faster replication method, with more performance impact on the primary.&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="1018964" author="dan@10gen.com" created="Tue, 1 Sep 2015 03:20:22 +0000"  >&lt;p&gt;Hi Baruch,&lt;br/&gt;
The index rebuilding on secondaries during initial sync &lt;b&gt;is&lt;/b&gt; done in parallel in v3.0.  This issue was resolved with &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9135&quot; title=&quot;fast index build for initial sync&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9135&quot;&gt;&lt;del&gt;SERVER-9135&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;There are a number of ways we&apos;re planning to improve the speed of initial sync.  Here&apos;s a few to watch: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-676&quot; title=&quot;use multiple cores for index sort-phase&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-676&quot;&gt;&lt;del&gt;SERVER-676&lt;/del&gt;&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18041&quot; title=&quot;Support parallel cloning during initial sync&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-18041&quot;&gt;SERVER-18041&lt;/a&gt;and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-16206&quot; title=&quot;Use the WT bulk loader for collection documents in cloner&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-16206&quot;&gt;&lt;del&gt;SERVER-16206&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="69777">SERVER-9135</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 1 Sep 2015 03:20:22 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        8 years, 24 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            8 years, 24 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>baruchoxman</customfieldvalue>
            <customfieldvalue>dan@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrkvtz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hscu13:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsbw4v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>