<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:19:36 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-28984] Improve Chunk Migration performance for clusters with high-latency Secondaries.</title>
                <link>https://jira.mongodb.org/browse/SERVER-28984</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;h2&gt;&lt;a name=&quot;TheProblem%3A&quot;&gt;&lt;/a&gt;The Problem:&lt;/h2&gt;
&lt;p&gt;For customers who have their availability zones geographically quite far away, secondaries can have an unavoidable high latency connection to the primary, even though the data-rates are well within needs.  Round-trip times as measured by ping can be in the 40-100ms range.&lt;/p&gt;

&lt;p&gt;In the above environment, chunk migration is terribly slow.  Chunk transfer rates in our cluster average a measly 4k bytes/sec.  Furthermore, range-deletes for removing the documents that belonged to a chunk that was just moved are also terribly slow.  The rest of Mongo works well in our cluster.  In fact, our secondaries&apos; Op-time lag are typically less than 1-second.  Standing up local secondary servers (i.e., close to the primary servers) improves chunk migration rates and range-delete rates by at least two orders of magnitude, but this has a tremendous cost, effectively doubling the number of secondary servers we require.&lt;/p&gt;

&lt;h2&gt;&lt;a name=&quot;APossibleSolution%3A&quot;&gt;&lt;/a&gt;A Possible Solution:&lt;/h2&gt;
&lt;p&gt;It appears from our experience that when the balancer calls moveChunk and/or rangeDeletes, it is using secondary-throttling by default with a write concern of at least 2.  &lt;font color=&quot;#d04437&quot;&gt;Why couldn&apos;t MongoDB support a &quot;high-latency-tolerant&quot; Chunk Migration mode; where calls to moveChunk and rangeDeletes for chunk migration, would use a write-concern of 1 (i.e., secondary-throttling disabled) except for the final write or delete?&lt;/font&gt;  &lt;br/&gt;
The final write or delete for each chunk could use secondary throttling with a write-concern of 2.  I believe this would yield a huge improvement in chunk migration performance for high-latency environments.  What are the down sides to this solution?  I really can&apos;t think of any.  &lt;/p&gt;</description>
                <environment></environment>
        <key id="377848">SERVER-28984</key>
            <summary>Improve Chunk Migration performance for clusters with high-latency Secondaries.</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="kaloian.manassiev@mongodb.com">Kaloian Manassiev</assignee>
                                    <reporter username="jimreitz">James Reitz</reporter>
                        <labels>
                    </labels>
                <created>Wed, 26 Apr 2017 18:12:01 +0000</created>
                <updated>Wed, 31 May 2017 21:23:23 +0000</updated>
                            <resolved>Sat, 29 Apr 2017 12:52:09 +0000</resolved>
                                    <version>3.2.11</version>
                                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>9</watches>
                                                                                                                <comments>
                            <comment id="1560879" author="kaloian.manassiev" created="Sat, 29 Apr 2017 12:51:33 +0000"  >&lt;p&gt;Thanks for confirming your server version &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jimreitz&quot; class=&quot;user-hover&quot; rel=&quot;jimreitz&quot;&gt;jimreitz&lt;/a&gt;. I am going to close this ticket as duplicate of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-23340&quot; title=&quot;Turn off moveChunk secondaryThrottle by default&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-23340&quot;&gt;&lt;del&gt;SERVER-23340&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Best regards,&lt;br/&gt;
-Kal.&lt;/p&gt;</comment>
                            <comment id="1560773" author="jimreitz" created="Sat, 29 Apr 2017 00:49:13 +0000"  >&lt;p&gt;@Kaloian Manassiev, I think you&apos;re right. We&apos;ve been using 3.2.11.  I didn&apos;t realize 3.4 had those changes.  &lt;/p&gt;

&lt;p&gt;Thanks @Dan Pasette, I&apos;ll keep that in mind.  Sometimes compression actually makes latency issues worse because it can add more latency to packet transmission.  But it&apos;s highly dependent on the implementation.&lt;/p&gt;</comment>
                            <comment id="1559613" author="dan@10gen.com" created="Thu, 27 Apr 2017 19:54:38 +0000"  >&lt;p&gt;Another improvement made in v3.4 was compression of the wire protocol for intra-cluster data transfer (&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3018&quot; title=&quot;Compression of wire protocol&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3018&quot;&gt;&lt;del&gt;SERVER-3018&lt;/del&gt;&lt;/a&gt;).  Especially in globally distributed clusters where bandwidth is limited (or expensive) this can be a big performance and cost-saving win.  Please note, that it is not enabled by default.  Docs are &lt;a href=&quot;https://docs.mongodb.com/manual/reference/program/mongod/#cmdoption-networkmessagecompressors&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="1559599" author="kaloian.manassiev" created="Thu, 27 Apr 2017 19:40:36 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jimreitz&quot; class=&quot;user-hover&quot; rel=&quot;jimreitz&quot;&gt;jimreitz&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;I believe what you are requesting duplicates &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-23340&quot; title=&quot;Turn off moveChunk secondaryThrottle by default&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-23340&quot;&gt;&lt;del&gt;SERVER-23340&lt;/del&gt;&lt;/a&gt;. Which version of the server are you using and which storage engine?&lt;/p&gt;

&lt;p&gt;Starting in version 3.4, if you are using the WiredTiger storage engine, the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.4.4/src/mongo/db/s/chunk_move_write_concern_options.cpp#L49&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;default&lt;/a&gt; write concern for migrations is w:1. The only time where majority write concern is used at the end of the migration.&lt;/p&gt;

&lt;p&gt;Historically, the default w:2 write concern was used as a way to achieve throttling and reduce the performance impact of migrations due to database level locking in MMAP V1.&lt;/p&gt;

&lt;p&gt;Best regards,&lt;br/&gt;
-Kal.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="274877">SERVER-23340</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 27 Apr 2017 19:40:36 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 41 weeks, 4 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>backlog-server-pm</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 41 weeks, 4 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>dan@mongodb.com</customfieldvalue>
            <customfieldvalue>jimreitz</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht6gef:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hsyp9r:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hs49un:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>