<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:07:44 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-5047] Better chunk selection when balancing</title>
                <link>https://jira.mongodb.org/browse/SERVER-5047</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;The balancer will always move the lowest chunk on the shard, &lt;del&gt;unless the highest chunk matches the lowest chunk on another shard&lt;/del&gt;.  Without any more information about data growth, this should probably be random or a smarter choice that minimizes fragmentation.  &lt;/p&gt;</description>
                <environment></environment>
        <key id="31307">SERVER-5047</key>
            <summary>Better chunk selection when balancing</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10033" iconUrl="https://jira.mongodb.org/images/icons/statuses/information.png" description="Status for tickets that need to be escalated and unblocked on our team.">Blocked</status>
                    <statusCategory id="4" key="indeterminate" colorName="inprogress"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-server-cluster-scalability">Backlog - Cluster Scalability</assignee>
                                    <reporter username="greg_10gen">Greg Studer</reporter>
                        <labels>
                            <label>oldshardingemea</label>
                    </labels>
                <created>Thu, 23 Feb 2012 01:54:52 +0000</created>
                <updated>Tue, 16 Jan 2024 14:19:38 +0000</updated>
                                                                            <component>Sharding</component>
                                        <votes>11</votes>
                                    <watches>24</watches>
                                                                                                                <comments>
                            <comment id="2072306" author="kevin.pulo@10gen.com" created="Mon, 26 Nov 2018 23:42:57 +0000"  >&lt;blockquote&gt;&lt;p&gt;unless the highest chunk matches the lowest chunk on another shard&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;This hasn&apos;t been the case for quite some time (at least 2.6), so I updated the description.&lt;/p&gt;</comment>
                            <comment id="175453" author="kamaradclimber" created="Tue, 16 Oct 2012 15:16:25 +0000"  >&lt;p&gt;a nice to have would be a plugable balancer strategy of course&lt;/p&gt;</comment>
                            <comment id="133268" author="kamaradclimber" created="Fri, 15 Jun 2012 13:42:09 +0000"  >&lt;p&gt;Hello,&lt;/p&gt;

&lt;p&gt;I have seen recent patches by Eliot on balancer going tag aware. Is there any progress on the thinking for improving balancer policy ?&lt;/p&gt;</comment>
                            <comment id="126132" author="kamaradclimber" created="Tue, 5 Jun 2012 06:43:39 +0000"  >&lt;p&gt;I agree that being able to choose strategy could be nice. However I can&apos;t think of a case where the default behavior would be better than the proposed patch.&lt;br/&gt;
The current implementation has serrious drawbacks and random shard assignmeent would probably behave better.&lt;br/&gt;
As for user provided strategy, it is very nice on paper but very error prone &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/smile.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/p&gt;</comment>
                            <comment id="126079" author="dchenbecker" created="Tue, 5 Jun 2012 04:11:14 +0000"  >&lt;p&gt;I agree with Neil. The default behavior is workable for many cases, but it would be really nice to be able to provide a custom selector for which chunk gets migrated from a given shard.&lt;/p&gt;</comment>
                            <comment id="126075" author="nsanch" created="Tue, 5 Jun 2012 04:03:53 +0000"  >&lt;p&gt;I don&apos;t think there exists a perfect chunk-assignment strategy that works for all use-cases. Could there be a set of strategies you can choose between?&lt;/p&gt;

&lt;p&gt;Alternatively, you could allow users to specify a snippet of JS that makes the choice, given the list of source chunks and dest chunks. The default behavior and Pierre&apos;s suggestion could both be implemented pretty easily in JS. This piece of code doesn&apos;t have to be all that performant, different people will want different things, the inputs are clearly defined, and the output is easily validated. Sounds perfect for JS?&lt;/p&gt;

&lt;p&gt;Pierre: nice code, btw.&lt;/p&gt;</comment>
                            <comment id="120076" author="linkfanel" created="Wed, 16 May 2012 15:05:28 +0000"  >&lt;p&gt;Alright. If you think of a particular use case and suspect or find that it yields bad results, feel free to let me know and I&apos;ll look into it.&lt;/p&gt;</comment>
                            <comment id="120033" author="eliot" created="Wed, 16 May 2012 12:47:14 +0000"  >&lt;p&gt;There are certain classes where this is good and others where it will make things worse.&lt;br/&gt;
So we need to test something like this across lots of various use cases.&lt;/p&gt;</comment>
                            <comment id="120026" author="linkfanel" created="Wed, 16 May 2012 12:22:30 +0000"  >&lt;p&gt;Could we have an update on this? We&apos;ve been running this in production on our cross-datacenter cluster with dozens of servers, and so far haven&apos;t seen any problem.&lt;/p&gt;</comment>
                            <comment id="115460" author="dchenbecker" created="Wed, 2 May 2012 04:11:51 +0000"  >&lt;p&gt;We&apos;re seeing sub-optimal Map/Reduce performance due to this issue. We have 5 shards in our cluster but for various sets of data the chunks are quite unbalanced. In one case almost 2.5M docs reside on two shards each, while two more shards have 400k and the final shard has zero docs. If someone with more knowledge thinks Pierre&apos;s patch looks good we would definitely consider using it.&lt;/p&gt;</comment>
                            <comment id="114806" author="linkfanel" created="Mon, 30 Apr 2012 13:10:55 +0000"  >&lt;p&gt;Here is my patch implementing this behavior in the balancer. It was tested and validated with real data. Please consider for inclusion.&lt;/p&gt;</comment>
                            <comment id="112617" author="linkfanel" created="Mon, 23 Apr 2012 15:30:54 +0000"  >&lt;p&gt;Chunk balancer logic test script&lt;/p&gt;</comment>
                            <comment id="112615" author="linkfanel" created="Mon, 23 Apr 2012 15:29:19 +0000"  >&lt;p&gt;Hello,&lt;/p&gt;

&lt;p&gt;I implemented a new algorithm for the pickChunk() function of the balancer. Instead of choosing the first chunk or a random chunk, it finds the biggest gap between two chunks of the receiver shard (i.e. with the most chunks of the donor shard between them) and picks a chunk in the middle of it. This tries to ensure that data is evenly distributed among shards across the key range, and fixes the issue Gr&#233;goire mentioned. It should address Benjamin&apos;s issue too, as, after shards are balanced, chunks to move will tend to be selected from where data is hot and growing.&lt;/p&gt;

&lt;p&gt;I&apos;m attaching a simulation script that I used to test and compare this behavior to the current one. For example, with 3 shards starting with all data on shard 1 only (. is a chunk on shard 1, O a chunk on shard 2 and X a chunk of shard 3, and each line is a balancing round) :&lt;/p&gt;

&lt;p&gt;&lt;tt&gt;..................................................&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;........................O.........................&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;........................OX........................&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;...........O............OX........................&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;...........O............OX...........X............&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;...........O............OX...........XO...........&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;...........OX...........OX...........XO...........&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.....O.....OX...........OX...........XO...........&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.....O.....OX.....X.....OX...........XO...........&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.....O.....OX.....X.....OX.....O.....XO...........&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.....O.....OX.....X.....OX.....O.....XO.....X.....&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.....O.....OX....OX.....OX.....O.....XO.....X.....&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;....XO.....OX....OX.....OX.....O.....XO.....X.....&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;....XO.....OX....OX.....OX.....O.....XO....OX.....&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;....XO.....OX....OX.....OX....XO.....XO....OX.....&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;....XO..O..OX....OX.....OX....XO.....XO....OX.....&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;....XO..O..OX....OX..X..OX....XO.....XO....OX.....&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;....XO..O..OX....OX..X..OX....XO..O..XO....OX.....&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;....XO..O..OX....OX..X..OX....XO..O..XO....OX..X..&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.O..XO..O..OX....OX..X..OX....XO..O..XO....OX..X..&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.O..XO.XO..OX....OX..X..OX....XO..O..XO....OX..X..&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.O..XO.XO..OX.O..OX..X..OX....XO..O..XO....OX..X..&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.O..XO.XO..OX.O..OX..X..OX.X..XO..O..XO....OX..X..&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.O..XO.XO..OX.O..OX.OX..OX.X..XO..O..XO....OX..X..&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.O..XO.XO..OX.O..OX.OX..OX.X..XO.XO..XO....OX..X..&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.O..XO.XO..OX.O..OX.OX..OX.X..XO.XO..XO.O..OX..X..&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.OX.XO.XO..OX.O..OX.OX..OX.X..XO.XO..XO.O..OX..X..&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.OX.XO.XO..OX.O..OX.OX..OX.X..XO.XO..XO.O..OX.OX..&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.OX.XO.XO..OX.OX.OX.OX..OX.X..XO.XO..XO.O..OX.OX..&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.OX.XO.XO..OX.OX.OX.OX..OX.XO.XO.XO..XO.O..OX.OX..&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.OX.XO.XO..OX.OX.OX.OX..OX.XO.XO.XO..XO.OX.OX.OX..&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.OX.XO.XOO.OX.OX.OX.OX..OX.XO.XO.XO..XO.OX.OX.OX..&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;.OX.XO.XOO.OX.OX.OX.OXX.OX.XO.XO.XO..XO.OX.OX.OX..&lt;/tt&gt;&lt;/p&gt;

&lt;p&gt;This is ugly :/ but run the script from a terminal to see other examples with shiny terminal colors!&lt;/p&gt;

&lt;p&gt;There is a minor flaw in this algorithm when adding several shards at a time with badly (not) distributed data (shown in test #5), new shards get chunks from separate parts of the key range only; but I don&apos;t see this as a real issue.&lt;/p&gt;

&lt;p&gt;I still need to test my patch with real mongod and mongos processes, but if you like this approach then I&apos;ll post it once it&apos;s ready.&lt;/p&gt;</comment>
                            <comment id="109643" author="kamaradclimber" created="Fri, 13 Apr 2012 16:13:31 +0000"  >&lt;p&gt;Looking at the current code, we&apos;ve found that the balancer is trying to minimize fragmentation of chunks (trying to move successive chunks to the same shards). Is there a reason for that objective ?&lt;/p&gt;</comment>
                            <comment id="109472" author="kamaradclimber" created="Fri, 13 Apr 2012 09:21:07 +0000"  >&lt;p&gt;My guess would that successive chunks (according to the shard key order) should not be on the same shard. This increase performance in case of sequential access or rouglhy targeted access patterns.&lt;br/&gt;
For instance, we have a compound shard key (partnerId , uniqueId) and we know that a given partnerId could become a hotspot for a small period of time. If consecutive chunks (having the same partnerId) are on different shards it will be balance the load on all shards.&lt;/p&gt;

&lt;p&gt;Random strategy work for most case, interlacing works better on mine.&lt;/p&gt;

&lt;p&gt;The current strategy is like no sharding at all for my set up : 3 shards. The first half of chunks are well balanced between 2 shards, the second half is all on the third one (the primary shard of the database of course).&lt;/p&gt;</comment>
                            <comment id="99991" author="bdarfler" created="Mon, 19 Mar 2012 00:10:22 +0000"  >&lt;p&gt;We have architected our system to use a shard key conforming to Kristina&apos;s suggestions in her blog post. (&lt;a href=&quot;http://www.snailinaturtleneck.com/blog/2011/01/04/how-to-choose-a-shard-key-the-card-game/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://www.snailinaturtleneck.com/blog/2011/01/04/how-to-choose-a-shard-key-the-card-game/&lt;/a&gt;). However, she has confirmed that this shard key won&apos;t work well with the balancer logic in 2.0. I want to make sure that this is addressed in the 2.2 release.&lt;/p&gt;

&lt;p&gt;The biggest pain point we have experienced with MongoDB has been balancing data. When cold chunks are moved by the balancer it requires significant i/o to read the data and eats away at RAM that could be better used to handle online queries. The benefit of Kristina&apos;s design is that chunks are split and moved when they are hot, addressing both of the issues with cold chunks. This is critical for online balancing without huge performance impacts. If we could simply add a new shard and hot data would start migrating over to it seamlessly we could avoid our seemingly never ending monthly crisis of adding shards and manually moving chunks to distribute load, or even worse doing a full dump and load of the data to balance everything out.&lt;/p&gt;

&lt;p&gt;Please, help make this experience better.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                                        </outwardlinks>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="416490">SERVER-30653</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="69626">SERVER-9114</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="69650">SERVER-9120</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="16133" name="balance.pl" size="4348" author="linkfanel" created="Mon, 23 Apr 2012 15:30:54 +0000"/>
                            <attachment id="16326" name="balancer.patch" size="1926" author="linkfanel" created="Mon, 30 Apr 2012 13:10:55 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>16.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="26583"><![CDATA[Cluster Scalability]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[5002K00000gk5b2QAA, 5002K00000kpfsuQAA]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 19 Mar 2012 00:10:22 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 11 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<a href='https://jira.mongodb.org/browse/PM-3213'>PM-3213</a>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-2324</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>dbeng-pm-bot</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 11 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-cluster-scalability</customfieldvalue>
            <customfieldvalue>bdarfler</customfieldvalue>
            <customfieldvalue>dchenbecker</customfieldvalue>
            <customfieldvalue>eliot</customfieldvalue>
            <customfieldvalue>kamaradclimber</customfieldvalue>
            <customfieldvalue>greg_10gen</customfieldvalue>
            <customfieldvalue>kevin.pulo@mongodb.com</customfieldvalue>
            <customfieldvalue>nsanch</customfieldvalue>
            <customfieldvalue>linkfanel</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hroco7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrfomn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4087</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hscp6f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>