<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:53:20 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-20156] Disk iops of balancer 10 times larger than simple insertions</title>
                <link>https://jira.mongodb.org/browse/SERVER-20156</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We want to know if balancer is efficiency enough to balance data when we add a new empty shard to a cluster&lt;/p&gt;

&lt;p&gt;The detail of the test&lt;/p&gt;

&lt;p&gt;mongodb version : 3.0.5&lt;br/&gt;
OS : centos 6&lt;/p&gt;

&lt;p&gt;We try to use tag aware shard tag to pre-allocate the chunk in shard2 and try to insert around 10 millions of records to it.&lt;/p&gt;

&lt;p&gt;&amp;#8211; The disk iops is around 10 during the insertion&lt;/p&gt;

&lt;p&gt;Then we remove the shard tag in shard2 and add it back to shard3 and let the balancer to move the 10 millions of records from shard2 to shard3 &lt;/p&gt;

&lt;p&gt;&amp;#8211; The disk iops is around 100 and more.  And the whole machine is slow down &lt;/p&gt;

&lt;p&gt;After the test, we are worrying that the entire cluster performance may drop if we try to add a empty shard to a cluster because a certain portion system resource is used to balance the data&lt;/p&gt;

&lt;p&gt;May I know if the improvement of balancer performance would be part of the roadmap ?&lt;/p&gt;</description>
                <environment></environment>
        <key id="227258">SERVER-20156</key>
            <summary>Disk iops of balancer 10 times larger than simple insertions</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="4" iconUrl="https://jira.mongodb.org/images/icons/priorities/minor.svg">Minor - P4</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="patrickwong@wisers.com">patrick wong</reporter>
                        <labels>
                    </labels>
                <created>Thu, 27 Aug 2015 01:40:26 +0000</created>
                <updated>Fri, 28 Aug 2015 21:27:49 +0000</updated>
                            <resolved>Thu, 27 Aug 2015 19:02:41 +0000</resolved>
                                    <version>3.0.5</version>
                                                    <component>Admin</component>
                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="1017468" author="ramon.fernandez" created="Fri, 28 Aug 2015 21:27:49 +0000"  >&lt;p&gt;The performance of the balancer is determined by the performance of your storage layer and the load on your servers and network. If a third of your data needs to be moved out the speed limitation will most likely come from storage IOPS and network bandwidth. &lt;a href=&quot;http://docs.mongodb.org/master/core/tag-aware-sharding/#tag-aware-sharding&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Tag aware sharding&lt;/a&gt; may help in some cases.&lt;/p&gt;

&lt;p&gt;That being said, you may be interested in looking at &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9120&quot; title=&quot;Load based balancing&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9120&quot;&gt;SERVER-9120&lt;/a&gt; and its related tickets.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                            <comment id="1016590" author="patrickwong@wisers.com" created="Fri, 28 Aug 2015 01:59:43 +0000"  >&lt;p&gt;Thanks for your reply.&lt;/p&gt;

&lt;p&gt;However, I&apos;m not request how to set a balancer window.&lt;/p&gt;

&lt;p&gt;For distributed computing, if it can&apos;t well distribute data among the node, the load of cluster can&apos;t share between different machines.&lt;/p&gt;

&lt;p&gt;If the distribution process is slow, adding machine can&apos;t help the performance immediately&lt;/p&gt;

&lt;p&gt;For example,  I have cluster of 3 shards and insertions is done on 24 * 7 basis&lt;/p&gt;

&lt;p&gt;After 1 year, I need to add a empty shard to improve the performance&lt;/p&gt;

&lt;p&gt;1/4 year data need to move the new shard&lt;/p&gt;

&lt;p&gt;If the balancer migration speed is slower 3 or 4 times than normal insertions, I may need 1 year to balance data.&lt;/p&gt;

&lt;p&gt;That may cause below issue&lt;/p&gt;

&lt;p&gt;1. I can&apos;t get immediate performance gain even I spend a new machine&lt;br/&gt;
2. The high disk io may cause existing cluster performance degrade.  Balancer window cannot help much if it is a 24*7 systems and the chunk migration lasts for 1 year&lt;/p&gt;

&lt;p&gt;As a result, I want to know if there is plan to improve the balancer performance and really hope it will be a valuable enhancement in coming releases&lt;/p&gt;</comment>
                            <comment id="1016275" author="ramon.fernandez" created="Thu, 27 Aug 2015 19:02:08 +0000"  >&lt;p&gt;Thanks for your report &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=patrickwong%40wisers.com&quot; class=&quot;user-hover&quot; rel=&quot;patrickwong@wisers.com&quot;&gt;patrickwong@wisers.com&lt;/a&gt;. The balancer needs to read data from disk to move it off to another shard, so depending on your configuration the disk may become the bottleneck here. You may want to investigate setting a &lt;a href=&quot;http://docs.mongodb.org/manual/tutorial/manage-sharded-cluster-balancer/#schedule-the-balancing-window&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;balancer window&lt;/a&gt; so migrations occur only times where I/O load will not impact production.&lt;/p&gt;

&lt;p&gt;Please note that the SERVER project is for reporting bugs or feature suggestions for the MongoDB server. For MongoDB-related support discussion please post on the &lt;a href=&quot;http://groups.google.com/group/mongodb-user&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;mongodb-user group&lt;/a&gt; or &lt;a href=&quot;http://stackoverflow.com/questions/tagged/mongodb&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Stack Overflow with the &lt;tt&gt;mongodb&lt;/tt&gt; tag&lt;/a&gt;, where your question will reach a larger audience. A question like this involving more discussion would be best posted on the &lt;a href=&quot;http://groups.google.com/group/mongodb-user&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;mongodb-user group&lt;/a&gt;. See also our &lt;a href=&quot;https://www.mongodb.org/about/support/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Technical Support page&lt;/a&gt; for additional support resources.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 27 Aug 2015 19:02:08 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        8 years, 24 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            8 years, 24 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>patrickwong@wisers.com</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrkw7j:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hscrb3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsfpov:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>