<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:05:45 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-4355] Support multiple parallel chunk migrations for the same collection</title>
                <link>https://jira.mongodb.org/browse/SERVER-4355</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;With larger clusters, one migration per cluster can be really really slow, a setting to allow the balancer to start one migration per machine may be more helpful in these cases.  &lt;/p&gt;</description>
                <environment></environment>
        <key id="25337">SERVER-4355</key>
            <summary>Support multiple parallel chunk migrations for the same collection</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="dianna.hohensee@mongodb.com">Dianna Hohensee</assignee>
                                    <reporter username="greg_10gen">Greg Studer</reporter>
                        <labels>
                    </labels>
                <created>Tue, 22 Nov 2011 21:25:20 +0000</created>
                <updated>Fri, 6 Apr 2018 01:12:34 +0000</updated>
                            <resolved>Wed, 20 Jul 2016 16:10:21 +0000</resolved>
                                                                    <component>Sharding</component>
                                        <votes>28</votes>
                                    <watches>36</watches>
                                                                                                                <comments>
                            <comment id="1857177" author="lucasoares" created="Fri, 6 Apr 2018 01:12:34 +0000"  >&lt;p&gt;Sorry.&lt;/p&gt;

&lt;p&gt;What I mean is the limit of parallel chunks migrations at shard level (multiple chunks migrations in the same shard).&lt;/p&gt;

&lt;p&gt;I expressed myself incorrectly. I mean is that would be nice to have not only one migration per shard but several migrations per shard.&lt;/p&gt;</comment>
                            <comment id="1857143" author="victorgp" created="Fri, 6 Apr 2018 00:47:28 +0000"  >&lt;p&gt;There are no chunk migrations at shard level... within a shard you have replication, not chunk migrations.&lt;/p&gt;</comment>
                            <comment id="1857136" author="lucasoares" created="Fri, 6 Apr 2018 00:31:59 +0000"  >&lt;p&gt;I don&apos;t think that issue should have been closed. For example I still have MANY migration issues when I add a new shard in my database (or remove a existing one). The migration is simply very slow because it was parallel only at cluster level, not shard level.&lt;/p&gt;

&lt;p&gt;I have SSD disks, 10Gbps network, 40 cores and MongoDB simply don&apos;t use my hardware resources to migrate data faster. Please let&apos;s try to improve chunks migrations.&lt;/p&gt;</comment>
                            <comment id="1330048" author="dianna.hohensee" created="Wed, 20 Jul 2016 16:10:21 +0000"  >&lt;p&gt;Closing this ticket as the task has been completed by &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-24853&quot; title=&quot;Refactor Balancer code to use MigrationManager in order to move chunks in parallel&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-24853&quot;&gt;&lt;del&gt;SERVER-24853&lt;/del&gt;&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1240937" author="blabergri@amadeus.com" created="Wed, 20 Apr 2016 07:59:14 +0000"  >&lt;p&gt;Agree with all comments here.&lt;br/&gt;
Parallelyze the balancing is really necessary. We encounters also the same problem with weeks of balancing before we can increase our collection size even if our machines have the capacity to handle more update traffic. It impact a lot our capacity planning. The more machine we add at a time and the more time we need to balance things.&lt;br/&gt;
A tunning of balancing capacity allowed would be good : &lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;number of chunk migration allowed in parallel per shard&lt;/li&gt;
	&lt;li&gt;or throughput limit&lt;br/&gt;
or why not do one migration per collection in parallel ?&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Our workaround is to create a new collection that we balanced manualy after having added shards and then refeed all data inside so we reduce the time to 2 days or 10 days in the worst case.&lt;br/&gt;
Any idea of parallelyzation would be good, not only one per shard but several migration per shard would be better.&lt;br/&gt;
Thanks &amp;amp; regards&lt;/p&gt;</comment>
                            <comment id="1109372" author="kevin.rice@searshc.com" created="Wed, 9 Dec 2015 14:56:43 +0000"  >&lt;p&gt;This is STILL a problem.  it&apos;s 2 years later.  Might want to work on this.&lt;/p&gt;

&lt;p&gt;Note that Cassandra has worked this out.  When you add a node to their cluster, it starts copying data immediately and the node is added in, like, maybe an hour or so. &lt;/p&gt;

&lt;p&gt;MongoDB cannot expand and shrink easily because this chunk migration thing is a giant huge massive hulking pain in the butt time-wise, it takes too long to move chunks.  In a recent deployment, it&apos;s slated to take WEEKS to migrate all the chunks.  This is unacceptable.  The only solution is to dump and reload data?   Excuse my frankness, but that&apos;s NOT an &apos;Enterprise Grade&apos; solution.&lt;/p&gt;</comment>
                            <comment id="278567" author="justanyone" created="Thu, 28 Feb 2013 22:52:53 +0000"  >&lt;p&gt;Limiting to one migration per machine can be troublesome since we may have different mount points per shard if there is more than one shard per machine.  &lt;/p&gt;

&lt;p&gt;IIRC, there&apos;s no need to be too smart about this with excessive locking.  In a 50 shard Mongo cluster, you&apos;d have 49 pairs of adjacent nodes, and a persistent daemon / subprocess / thread could be spawned to handle the balancing of those two nodes.  Every once in a while (minute?  5 minutes?), the daemon could wake up, compare chunk counts, and either go to sleep or go to work.  &lt;/p&gt;

&lt;p&gt;Visibility of the balancer state is of significant concern.  Therefore, I&apos;d suggest the balancer write to a capped collection with what each node-pair balancer is doing, indexed by &apos;fromShard:toShard&apos; or something.&lt;/p&gt;

&lt;p&gt;Part of the problem here is having balancers on mongos routers; it seems like this code belongs somewhere else, in an independent process, with parameters for how aggressive it should be as well as the existing ones of which hours it should run, etc.&lt;/p&gt;

&lt;p&gt;Aggressiveness is important because if we&apos;re writing to a single shard so much that there is no spare IO available for a long time, it can never migrate work to other shards to balance the load better.  Some kind of radioactive-decay curve should apply so that the longer it goes unbalanced, the more important balancing is, until finally balancing is more important than writing new data and you can get a chunk actually moved despite the overload.&lt;/p&gt;

&lt;p&gt;Just some ideas.&lt;/p&gt;




</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="297836">SERVER-24853</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="100555">SERVER-11905</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="265484">SERVER-22616</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="265917">SERVER-22669</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="27068">SERVER-4537</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[500A000000UaVHyIAN, 500A000000UaXEYIA3]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 28 Feb 2013 22:52:53 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 44 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-229</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>lucasoares</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 44 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>blabergri@amadeus.com</customfieldvalue>
            <customfieldvalue>dianna.hohensee@mongodb.com</customfieldvalue>
            <customfieldvalue>greg_10gen</customfieldvalue>
            <customfieldvalue>justanyone</customfieldvalue>
            <customfieldvalue>kevin.rice@searshc.com</customfieldvalue>
            <customfieldvalue>lucasoares</customfieldvalue>
            <customfieldvalue>victorgp</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hroktz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrcqtb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4825</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="1015">Sharding 17 (07/15/16)</customfieldvalue>
    <customfieldvalue id="1016">Sharding 18 (08/05/16)</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsbwbr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>