<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:07:47 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-67301] Balancer may perform one unnecessary migration for a completely balanced collection</title>
                <link>https://jira.mongodb.org/browse/SERVER-67301</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;h4&gt;&lt;a name=&quot;Preamble&quot;&gt;&lt;/a&gt;Preamble&lt;/h4&gt;

&lt;p&gt;As part of a range deletion:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;A &lt;a href=&quot;https://www.mongodb.com/docs/manual/reference/parameters/#mongodb-parameter-param.rangeDeleterBatchSize&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;batch&lt;/a&gt; of documents belonging to the orphaned range &lt;a href=&quot;https://github.com/mongodb/mongo/blob/ac297836aede311622e60ee502dda7ac6bb344fc/src/mongo/db/s/range_deletion_util.cpp#L327-L331&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;is deleted&lt;/a&gt; (each document in the batch is deleted in a single delete).&lt;/li&gt;
	&lt;li&gt;The counter of orphans &lt;a href=&quot;https://github.com/mongodb/mongo/blob/ac297836aede311622e60ee502dda7ac6bb344fc/src/mongo/db/s/range_deletion_util.cpp#L332-L333&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;is updated&lt;/a&gt; accordingly, triggering also an &lt;a href=&quot;https://github.com/mongodb/mongo/blob/114313e06a03cfc19a502209c525330a29c36be7/src/mongo/db/s/migration_util.cpp#L716&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;update of the orphans counter on the BalancerStatsRegistry&lt;/a&gt;.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;Note that 1-2 are not happening in the same storage transaction, and - even if they would - the update of the in-memory stats registry would still not happen atomically.&lt;/p&gt;

&lt;p&gt;The balancer is considering the cluster balanced if the following condition is met:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;size(most loaded shard) &amp;lt;= size(least loaded shard) + 2 * chunkSize&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;&#160;&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Problem&quot;&gt;&lt;/a&gt;Problem&lt;/h4&gt;

&lt;p&gt;If the range deletion happens on an already balanced collection while a range deletion is ongoing, it may happen that the balancer retrieves statistics from shards between bullets 1 and 2. This may trigger an unnecessary migration (no big deal as the chunk would be later migrated back, but still it&apos;s an unnecessary operation).&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Example&quot;&gt;&lt;/a&gt;Example&lt;/h4&gt;

&lt;p&gt;Given a chunk size of &lt;b&gt;128MB&lt;/b&gt; for a given collection, if the collection size between between two shards A and B differs less than &lt;b&gt;256MB&lt;/b&gt; the balancer should not move any data.&lt;/p&gt;


&lt;p&gt;However, it could happen the following:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;ShardA has &lt;b&gt;400MB&lt;/b&gt;, ShardB has &lt;b&gt;0MB&lt;/b&gt;&lt;/li&gt;
	&lt;li&gt;Move a range of &lt;b&gt;128MB&lt;/b&gt; from A to B&lt;/li&gt;
	&lt;li&gt;Move a range of &lt;b&gt;128MB&lt;/b&gt; from A to B&lt;/li&gt;
	&lt;li&gt;ShardB receives a bit of inserts, let&apos;s say &lt;b&gt;24MB&lt;/b&gt;&lt;/li&gt;
&lt;/ul&gt;



&lt;p&gt;At this point the collection is balanced because &lt;em&gt;(280 - 144 = 136)&lt;/em&gt; is less than (&lt;em&gt;2 * chunkSize = 256&lt;/em&gt;) :&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;ShardA still has &lt;b&gt;400MB&lt;/b&gt;: &lt;b&gt;144MB&lt;/b&gt; (actual data size) plus &lt;b&gt;256MB&lt;/b&gt; (orphans)&lt;/li&gt;
	&lt;li&gt;ShardB has &lt;b&gt;280MB&lt;/b&gt; (a bit more of 256MB, accounting also for some inserts)&lt;/li&gt;
&lt;/ul&gt;



&lt;p&gt;The range deleter kicks in:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Starts deleting the orphans for one range...&lt;/li&gt;
	&lt;li&gt;The range deleter has so far deleted &lt;b&gt;125MB&lt;/b&gt;&lt;/li&gt;
	&lt;li&gt;The balancer asks all the shards what&apos;s their data size:
	&lt;ul&gt;
		&lt;li&gt;Shard A will reply (&lt;b&gt;400&lt;/b&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;original size&amp;#93;&lt;/span&gt; - &lt;b&gt;125&lt;/b&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;docs already deleted&amp;#93;&lt;/span&gt;) - (&lt;b&gt;128 * 2&lt;/b&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;tracked number of orphaned docs&amp;#93;&lt;/span&gt;)&lt;br/&gt;
So it replies &lt;b&gt;400 - 125 - 256 = 19MB&lt;/b&gt;&lt;/li&gt;
		&lt;li&gt;Shard B will reply &lt;b&gt;280MB&lt;/b&gt;&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;The balancer performs its calculation on the chunkSize: (&lt;em&gt;280 - 19 &amp;gt; 2 * chunkSize)&lt;/em&gt; ? &lt;b&gt;YES&lt;/b&gt; because (&lt;em&gt;261 &amp;gt; 256)&lt;/em&gt;&lt;/li&gt;
	&lt;li&gt;A range is wrongly moved from shard B to shard A&lt;/li&gt;
&lt;/ul&gt;
</description>
                <environment></environment>
        <key id="2069160">SERVER-67301</key>
            <summary>Balancer may perform one unnecessary migration for a completely balanced collection</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="silvia.surroca@mongodb.com">Silvia Surroca</assignee>
                                    <reporter username="pierlauro.sciarelli@mongodb.com">Pierlauro Sciarelli</reporter>
                        <labels>
                            <label>shardingemea-qw</label>
                    </labels>
                <created>Wed, 15 Jun 2022 13:23:05 +0000</created>
                <updated>Sun, 29 Oct 2023 21:36:47 +0000</updated>
                            <resolved>Wed, 27 Jul 2022 09:48:07 +0000</resolved>
                                                    <fixVersion>6.0.3</fixVersion>
                    <fixVersion>6.1.0-rc0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="4903360" author="xgen-internal-githook" created="Fri, 14 Oct 2022 13:35:20 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Silvia Surroca&apos;, &apos;email&apos;: &apos;silvia.surroca@mongodb.com&apos;, &apos;username&apos;: &apos;silviasuhu&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-67301&quot; title=&quot;Balancer may perform one unnecessary migration for a completely balanced collection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-67301&quot;&gt;&lt;del&gt;SERVER-67301&lt;/del&gt;&lt;/a&gt; Balancer may perform one unnecessary migration for a completely balanced collection&lt;br/&gt;
Branch: v6.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/74d6780ecbd503e6b5e762d1939de9c2a7654f9e&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/74d6780ecbd503e6b5e762d1939de9c2a7654f9e&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4707905" author="xgen-internal-githook" created="Wed, 27 Jul 2022 09:21:17 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Silvia Surroca&apos;, &apos;email&apos;: &apos;silvia.surroca@mongodb.com&apos;, &apos;username&apos;: &apos;silviasuhu&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-67301&quot; title=&quot;Balancer may perform one unnecessary migration for a completely balanced collection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-67301&quot;&gt;&lt;del&gt;SERVER-67301&lt;/del&gt;&lt;/a&gt; Balancer may perform one unnecessary migration for a completely balanced collection&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/4c4fe953305a928228a0f3d320092ae8929255a3&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/4c4fe953305a928228a0f3d320092ae8929255a3&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="2065908">SERVER-67171</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10520">
                    <name>Problem/Incident</name>
                                            <outwardlinks description="causes">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="2112238">SERVER-68777</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23470"><![CDATA[v6.0]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 23 Jun 2022 10:23:28 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 16 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-2323</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 16 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>35.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>pierlauro.sciarelli@mongodb.com</customfieldvalue>
            <customfieldvalue>silvia.surroca@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0yoyn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i0hk4w:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="6285">Sharding EMEA 2022-07-11</customfieldvalue>
    <customfieldvalue id="6286">Sharding EMEA 2022-07-25</customfieldvalue>
    <customfieldvalue id="6287">Sharding EMEA 2022-08-08</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10555" key="com.atlassian.jira.plugin.system.customfieldtypes:float">
                        <customfieldname>Story Points</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0yb3z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>