<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:24:11 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-10833] Initial Sync Performance</title>
                <link>https://jira.mongodb.org/browse/SERVER-10833</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;After multiple harddisk failures on the secondary member of a replica set we are desperatly trying bring a new member up and running without taking down the whole system and manually copying the files from the primary&lt;/p&gt;

&lt;p&gt;after mongo completed copying the data, the sync process is more or less stuck on creating the indices.&lt;br/&gt;
this process already took almost 1 week! and is still not finished yet&lt;/p&gt;

&lt;p&gt;any error or loss of connection to the primary in the sync process will restart the whole process from the beginning, deleting all previously synced data and indices&lt;/p&gt;

&lt;p&gt;log excerpt:&lt;/p&gt;

&lt;p&gt;2013-09-20T11:22:07.010835+02:00 rack4-5 mongod.30000&lt;span class=&quot;error&quot;&gt;&amp;#91;25590&amp;#93;&lt;/span&gt;: Fri Sep 20 11:22:07.010 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsSync&amp;#93;&lt;/span&gt; #011#011Index: (2/3) BTree Bottom Up Progress: 3141451300/3764787709#01183%&lt;br/&gt;
2013-09-20T11:22:17.005409+02:00 rack4-5 mongod.30000&lt;span class=&quot;error&quot;&gt;&amp;#91;25590&amp;#93;&lt;/span&gt;: Fri Sep 20 11:22:17.004 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsSync&amp;#93;&lt;/span&gt; #011#011Index: (2/3) BTree Bottom Up Progress: 3141577600/3764787709#01183%&lt;br/&gt;
2013-09-20T11:22:27.010042+02:00 rack4-5 mongod.30000&lt;span class=&quot;error&quot;&gt;&amp;#91;25590&amp;#93;&lt;/span&gt;: Fri Sep 20 11:22:27.009 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsSync&amp;#93;&lt;/span&gt; #011#011Index: (2/3) BTree Bottom Up Progress: 3141705300/3764787709#01183%&lt;/p&gt;

&lt;p&gt;Size of the Data is currently at 2.4TB, increasing at a rate of about 20GB per day&lt;/p&gt;

&lt;p&gt;is there any way to speed up this process?&lt;br/&gt;
is there any way to have some sort of incremental initial sync,since in case of an error, having to restart the whole process from the beginning is utterly insane&lt;br/&gt;
why do indices have to be rebuilt from scratch instead of just getting copied from the primary anyway&lt;/p&gt;

&lt;p&gt;currently mongodb is basically unusable for our purpose, because in the event of a hardware failure, we are unable to get a replacement back online in a reasonable amount of time without shutting down the whole database&lt;/p&gt;</description>
                <environment>OpenSuse x64</environment>
        <key id="90925">SERVER-10833</key>
            <summary>Initial Sync Performance</summary>
                <type id="6" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14720&amp;avatarType=issuetype">Question</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="appelhoff@kajomi.de">Winand Appelhoff</reporter>
                        <labels>
                    </labels>
                <created>Fri, 20 Sep 2013 14:46:16 +0000</created>
                <updated>Fri, 3 Jan 2014 04:12:26 +0000</updated>
                            <resolved>Mon, 4 Nov 2013 21:32:09 +0000</resolved>
                                    <version>2.2.1</version>
                    <version>2.4.6</version>
                                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="429063" author="scotthernandez" created="Fri, 20 Sep 2013 15:13:03 +0000"  >&lt;p&gt;It is best to ask these kinds of question on the mongodb-user list (&lt;a href=&quot;http://groups.google.com/forum/#!forum/mongodb-user&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://groups.google.com/forum/#!forum/mongodb-user&lt;/a&gt;). This project is meant for bugs and feature requests.&lt;/p&gt;

&lt;p&gt;Building indexes is done as quickly as possible and there is no user tuning options, aside from having more memory or faster storage. &lt;/p&gt;

&lt;p&gt;Replication works at the operation (logical) level and not at the block or file level so each member must create its own storage files &amp;#8211; this model allows having different version and storage formats on different members.&lt;/p&gt;

&lt;p&gt;It is best to use a filesystem backup or copy of the dbpath files to bring up a new member, to reduce the time and work done, as described in the docs here: &lt;a href=&quot;http://docs.mongodb.org/manual/tutorial/resync-replica-set-member/#replica-set-resync-by-copying&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://docs.mongodb.org/manual/tutorial/resync-replica-set-member/#replica-set-resync-by-copying&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;If you have a recent backup which you can use to start the new secondary from I would suggest using it. As long as the backup was made during the current oplog windows of operation it will be able to replicate and catch up.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 20 Sep 2013 15:13:03 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        10 years, 21 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            10 years, 21 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>scotthernandez</customfieldvalue>
            <customfieldvalue>appelhoff@kajomi.de</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrmfz3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrt0tr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>83214</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hspauf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>