<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:47:59 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-59729] Investigate to find the best default values for FCBIS parameters</title>
                <link>https://jira.mongodb.org/browse/SERVER-59729</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;server parameters:&lt;br/&gt;
fileBasedInitialSyncMaxLagSec &lt;br/&gt;
fileBasedInitialSyncMaxCyclesWithoutProgress&lt;/p&gt;</description>
                <environment></environment>
        <key id="1865655">SERVER-59729</key>
            <summary>Investigate to find the best default values for FCBIS parameters</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="matthew.russotto@mongodb.com">Matthew Russotto</assignee>
                                    <reporter username="m.maher@mongodb.com">Moustafa Maher</reporter>
                        <labels>
                    </labels>
                <created>Wed, 1 Sep 2021 20:18:57 +0000</created>
                <updated>Sun, 29 Oct 2023 21:48:58 +0000</updated>
                            <resolved>Sat, 4 Dec 2021 00:08:25 +0000</resolved>
                                                    <fixVersion>5.2.0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="4230816" author="xgen-internal-githook" created="Fri, 3 Dec 2021 23:58:43 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Matthew Russotto&apos;, &apos;email&apos;: &apos;matthew.russotto@mongodb.com&apos;, &apos;username&apos;: &apos;mtrussotto&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-59729&quot; title=&quot;Investigate to find the best default values for FCBIS parameters&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-59729&quot;&gt;&lt;del&gt;SERVER-59729&lt;/del&gt;&lt;/a&gt; Investigate to find the best default values for FCBIS parameters&lt;/p&gt;

&lt;p&gt;We have a preliminary result of 69.3 GB (14.3MB/sec) in 4814 seconds for the initialsync logkeeper&lt;br/&gt;
workload. We also have a max insert performance on the order of 600KB/sec from our performance&lt;br/&gt;
testing.&lt;/p&gt;

&lt;p&gt;These will vary considerably with hardware and load, but these seem like reasonable numbers to start&lt;br/&gt;
with. If we assume a large database (1000GB), we will transfer it in about 70000 seconds, in which&lt;br/&gt;
time we could have inserted another 42GB of data. We&apos;ll transfer that in about 3000 seconds,&lt;br/&gt;
inserting 1.8GB of data, which will transfer in 125 seconds (we&apos;re transferring only oplog journal&lt;br/&gt;
files, so no write amplification). So 3 cycles should bring us within a few minutes even under heavy&lt;br/&gt;
load. We can set fileBasedInitialSyncMaxLagSec to 300 (5 minutes) and&lt;br/&gt;
fileBasedInitialSyncMaxCyclesWithoutProgress to 3.&lt;/p&gt;

&lt;p&gt;Setting fileBasedInitialSyncMaxLagSec much smaller likely doesn&apos;t help, because we cannot be doing&lt;br/&gt;
oplog application while downloading in FCBIS (once we write to the data store through WT we cannot&lt;br/&gt;
add log files); any time we spend in oplog application will have to be made up after the node begins&lt;br/&gt;
normal operation.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/df86cb8567b38302ac1ec616566523c05182888c&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/df86cb8567b38302ac1ec616566523c05182888c&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4227279" author="matthew.russotto" created="Thu, 2 Dec 2021 16:34:37 +0000"  >&lt;p&gt;We have a preliminary result of 69.3 GB (14.3MB/sec) in 4814 seconds for the initialsync logkeeper workload.  We also have a max insert performance on the order of 600KB/sec from our performance testing (not far from a very recent doc here:&lt;br/&gt;
&lt;a href=&quot;https://www.mongodb.com/developer/how-to/mongodb-network-compression/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://www.mongodb.com/developer/how-to/mongodb-network-compression/&lt;/a&gt; )&lt;/p&gt;

&lt;p&gt;These will vary considerably with hardware and load, but these seem like reasonable numbers to start with.  If we assume a large database (1000GB), we will transfer it in about 70000 seconds, in which time we could have inserted another 42GB of data.  We&apos;ll transfer that in about 3000 seconds, inserting 1.8GB of data, which will transfer in 125 seconds (we&apos;re transferring only oplog journal files, so no write amplification).  So 3 cycles should bring us within a few minutes even under heavy load.  We can set fileBasedInitialSyncMaxLagSec to 300 (5 minutes) and fileBasedInitialSyncMaxCyclesWithoutProgress to 3.  &lt;/p&gt;

&lt;p&gt;Setting fileBasedInitialSyncMaxLagSec much smaller likely doesn&apos;t help, because we cannot be doing oplog application while downloading in FCBIS (once we write to the data store through WT we cannot add log files); any time we spend in oplog application will have to be made up after the node begins normal operation.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 2 Dec 2021 16:34:37 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 9 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1897</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 9 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>matthew.russotto@mongodb.com</customfieldvalue>
            <customfieldvalue>m.maher@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i005sv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hzk5hz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="5528">Replication 2021-11-15</customfieldvalue>
    <customfieldvalue id="5529">Replication 2021-11-29</customfieldvalue>
    <customfieldvalue id="5530">Replication 2021-12-13</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzzs07:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>