<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:36:27 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-55430] Record metrics about whether a collection is rebalanced after resharding op finishes </title>
                <link>https://jira.mongodb.org/browse/SERVER-55430</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Resharding attempts to create an initial chunk distribution such that the collection will not be rebalanced by the balancer immediately after (absent of any topology changes, etc of course). It would be useful to collect metrics on whether the balancer rebalances a collection that has just been resharded in order to determine whether resharding&apos;s initial split policy is in fact creating a good initial distribution.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1655661">SERVER-55430</key>
            <summary>Record metrics about whether a collection is rebalanced after resharding op finishes </summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="randolph@mongodb.com">Randolph Tan</assignee>
                                    <reporter username="janna.golden@mongodb.com">Janna Golden</reporter>
                        <labels>
                    </labels>
                <created>Mon, 22 Mar 2021 20:18:45 +0000</created>
                <updated>Sun, 29 Oct 2023 21:55:54 +0000</updated>
                            <resolved>Wed, 23 Jun 2021 20:48:41 +0000</resolved>
                                                    <fixVersion>5.0.0-rc3</fixVersion>
                    <fixVersion>5.1.0-rc0</fixVersion>
                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="4108109" author="JIRAUSER1259052" created="Wed, 6 Oct 2021 18:57:17 +0000"  >&lt;p&gt;Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it&#8217;s been triggered. For more active release information, please keep an eye on #server-release. Thank you!&lt;/p&gt;</comment>
                            <comment id="3881915" author="xgen-internal-githook" created="Wed, 16 Jun 2021 17:49:28 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Randolph Tan&apos;, &apos;email&apos;: &apos;randolph@10gen.com&apos;, &apos;username&apos;: &apos;renctan&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-55430&quot; title=&quot;Record metrics about whether a collection is rebalanced after resharding op finishes &quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-55430&quot;&gt;&lt;del&gt;SERVER-55430&lt;/del&gt;&lt;/a&gt; Record metrics about whether a collection is rebalanced after resharding op finishes&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 5c00024e3cf4a27039117e000e475c6ee797c700)&lt;br/&gt;
Branch: v5.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/dfdbeaa9f6bc166c69b9a4de9a538c08952b8739&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/dfdbeaa9f6bc166c69b9a4de9a538c08952b8739&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3881687" author="xgen-internal-githook" created="Wed, 16 Jun 2021 16:32:36 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Randolph Tan&apos;, &apos;email&apos;: &apos;randolph@10gen.com&apos;, &apos;username&apos;: &apos;renctan&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-55430&quot; title=&quot;Record metrics about whether a collection is rebalanced after resharding op finishes &quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-55430&quot;&gt;&lt;del&gt;SERVER-55430&lt;/del&gt;&lt;/a&gt; Record metrics about whether a collection is rebalanced after resharding op finishes&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/5c00024e3cf4a27039117e000e475c6ee797c700&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/5c00024e3cf4a27039117e000e475c6ee797c700&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3857451" author="janna.golden" created="Thu, 3 Jun 2021 14:45:47 +0000"  >&lt;p&gt;Hmm, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=renctan&quot; class=&quot;user-hover&quot; rel=&quot;renctan&quot;&gt;renctan&lt;/a&gt; that&apos;s a good point about it being difficult to know what collections have been resharded recently. I think &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=max.hirschhorn&quot; class=&quot;user-hover&quot; rel=&quot;max.hirschhorn&quot;&gt;max.hirschhorn&lt;/a&gt;&apos;s idea about recording it in the ReshardingMetrics makes sense, we&apos;d essentially just want to check that the initial split alg is putting this &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c03426c28f007425e0aa7de65e3ca9509624128a/src/mongo/db/s/balancer/balancer_policy.cpp#L537-L538&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&quot;ideal number of chunks per shard&lt;/a&gt;. I think it might even make sense to do it when we create the chunks because we have access to the shard info (what zones are associated with what shards) there as well, though I don&apos;t know if we have access to the ReshardingMetrics at that point.&lt;/p&gt;</comment>
                            <comment id="3855368" author="max.hirschhorn@10gen.com" created="Wed, 2 Jun 2021 23:03:23 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=renctan&quot; class=&quot;user-hover&quot; rel=&quot;renctan&quot;&gt;renctan&lt;/a&gt;, I think a per&amp;#45;collection metric could make it difficult to get reporting on because per&amp;#45;collection metrics aren&apos;t something we could add to a serverStatus section tracked by FTDC since would lead to excessive schema changes (e.g. as what happened with the range deleter and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-47641&quot; title=&quot;Limit size of serverStatus metrics for the range deleter&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-47641&quot;&gt;&lt;del&gt;SERVER-47641&lt;/del&gt;&lt;/a&gt;). My take on &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-55430&quot; title=&quot;Record metrics about whether a collection is rebalanced after resharding op finishes &quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-55430&quot;&gt;&lt;del&gt;SERVER-55430&lt;/del&gt;&lt;/a&gt; would be to have some part of ReshardingCoordinator call the function for &apos;is this collection balanced?&apos; that the Balancer thread would normally call and record the yes/no answer in the ReshardingMetrics before completing the operation. One step further could be to record how unbalanced the collection is. I would want to aim for a single number summarization (maybe [max nChunks - min nChunks]?) to avoid inducing schema changes from a per&amp;#45;shard metric either.&lt;/p&gt;</comment>
                            <comment id="3854656" author="renctan" created="Wed, 2 Jun 2021 18:54:17 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=max.hirschhorn&quot; class=&quot;user-hover&quot; rel=&quot;max.hirschhorn&quot;&gt;max.hirschhorn&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=janna.golden&quot; class=&quot;user-hover&quot; rel=&quot;janna.golden&quot;&gt;janna.golden&lt;/a&gt;. Here are my proposal for this ticket, what do you guys think?&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;Since it would be hard for the balancer to distinguish which collection has just been resharded, what if we just have stats for all sharded collections?&lt;/li&gt;
	&lt;li&gt;The most basic stat I can think of is standard deviation of chunks per tag/zone per collection. I&apos;m currently thinking of putting a new field in config.collections to contain this stats, and collStats would display this info.&lt;/li&gt;
	&lt;li&gt;The question now becomes, when do we update the stats? The easiest would be at the beginning of the balancer round, because that is when the balancer collects stats about chunk distribution and it doesn&apos;t update it after a migration. This would achieve the goal of exposing chunk imbalance caused by resharding, but it can also be a weird stat to show, since it is somewhat &quot;delayed&quot; and does not represent the most up to date distribution as the balancer round progresses.&lt;/li&gt;
&lt;/ol&gt;
</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10320">
                    <name>Documented</name>
                                                                <inwardlinks description="is documented by">
                                        <issuelink>
            <issuekey id="1780705">DOCS-14565</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="2333673">SERVER-76890</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="21777"><![CDATA[v5.0]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 2 Jun 2021 18:54:17 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 18 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_17052" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Downstream Changes Summary</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>New field added to serverStatus metrics under shardingStatistics.resharding: lastOpEndingChunkImbalance. This field contains the largest diff (maxNumChunksInShard - minNumChunksInShard) among all zones for the collection that went through the last resharding operation. This is only update on the config servers.</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16942"><![CDATA[Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-234</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 18 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>janna.golden@mongodb.com</customfieldvalue>
            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>
            <customfieldvalue>randolph@mongodb.com</customfieldvalue>
            <customfieldvalue>vivian.ge@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hz0mi7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hylu07:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4519">Sharding 2021-06-14</customfieldvalue>
    <customfieldvalue id="4520">Sharding 2021-06-28</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10555" key="com.atlassian.jira.plugin.system.customfieldtypes:float">
                        <customfieldname>Story Points</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_17051" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Teams Impacted</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16944"><![CDATA[Docs]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hz08rb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>