<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:48:18 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-38219] dataSize, collStats diagnostic commands not detecting change of shards after a sharded collection is dropped</title>
                <link>https://jira.mongodb.org/browse/SERVER-38219</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;It seems the dataSize and collStats commands rely on the setShardVersion mechanism to be triggered by CRUD traffic coming through the same mongos node to flush the metadata, and will be incorrect until one of those happen, or a flushRouterConfig command is run.&lt;/p&gt;

&lt;p&gt;This caught me out recently as a mongos node used exclusively by the DBAs didn&apos;t reflect changes made by the app team on the app server&apos;s mongos nodes. All the normal queries and updates were being done by those apps, whereas I was only doing diagnosis (no CRUD ops). For half a day I investigated an imaginary data imbalance issue the dataSize and collStat commands were showing me, only to find as soon as I ran a flushRouterConfig the issue was gone. I think doing a single CRUD op on the collection in question also resolves it.&lt;/p&gt;

&lt;p&gt;The production event preceding the issue was the dropping of a big sharded collection and recreating it with a new shard key, but presumably mongos nodes will also be stale for other metadata changes such as chunk moves.&lt;/p&gt;

&lt;p&gt;Issue was encountered in 3.6.7, but so far as I can see 4.0 code for these commands (and maybe all non-CRUD commands?) is using the catalog cache class in the same way, so I suspect it&apos;s still an issue for current release versions too.&lt;/p&gt;</description>
                <environment></environment>
        <key id="636920">SERVER-38219</key>
            <summary>dataSize, collStats diagnostic commands not detecting change of shards after a sharded collection is dropped</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="cheahuychou.mao@mongodb.com">Cheahuychou Mao</assignee>
                                    <reporter username="akira.kurogane@gmail.com">&#31456; &#40658;&#37444;</reporter>
                        <labels>
                            <label>ShardingRoughEdges</label>
                            <label>sharding-wfbf-day</label>
                    </labels>
                <created>Wed, 21 Nov 2018 05:25:49 +0000</created>
                <updated>Mon, 15 Nov 2021 17:02:33 +0000</updated>
                            <resolved>Tue, 14 Apr 2020 17:14:48 +0000</resolved>
                                    <version>3.6.7</version>
                                                    <component>Sharding</component>
                                        <votes>1</votes>
                                    <watches>15</watches>
                                                                                                                <comments>
                            <comment id="3039962" author="akira.kurogane@gmail.com" created="Wed, 15 Apr 2020 00:09:32 +0000"  >&lt;p&gt;Thanks for update. Sounds resolved. Just glad to hear it&apos;s resolved in master branch, getting backported to v4.0 is a nice extra.&lt;/p&gt;</comment>
                            <comment id="3039133" author="chou.mao" created="Tue, 14 Apr 2020 16:45:16 +0000"  >&lt;p&gt;Closing this ticket since &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-47436&quot; title=&quot;Make shards validate shardKey in dataSize command&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-47436&quot;&gt;&lt;del&gt;SERVER-47436&lt;/del&gt;&lt;/a&gt; was done and backported all the way to v4.0. Unfortunately,  dataSize command in v3.6 does not use shard versioning so a flushRouterConfig or a CRUD command is required to make the stale mongos refresh.&lt;/p&gt;</comment>
                            <comment id="2876436" author="jason.carey" created="Thu, 13 Feb 2020 14:12:43 +0000"  >&lt;p&gt;Adding to a wfbf day to investigate why versioning doesn&apos;t cover this case&lt;/p&gt;</comment>
                            <comment id="2133908" author="alexey.menshikov" created="Thu, 31 Jan 2019 20:17:49 +0000"  >&lt;p&gt;I was able to reproduce both issues on 3.6.7 and 4.0.5&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="2068684" author="thomas.schubert" created="Wed, 21 Nov 2018 05:36:38 +0000"  >&lt;p&gt;Thanks for the detailed report, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=akira&quot; class=&quot;user-hover&quot; rel=&quot;akira&quot;&gt;akira&lt;/a&gt;!&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="1309659">SERVER-47436</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="201598" name="repro.sh" size="3562" author="akira" created="Wed, 21 Nov 2018 05:30:32 +0000"/>
                            <attachment id="201599" name="repro_result.txt" size="1943" author="akira" created="Wed, 21 Nov 2018 05:32:38 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 21 Nov 2018 05:36:38 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 43 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 43 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>alexey.menshikov@mongodb.com</customfieldvalue>
            <customfieldvalue>cheahuychou.mao@mongodb.com</customfieldvalue>
            <customfieldvalue>kelsey.schubert@mongodb.com</customfieldvalue>
            <customfieldvalue>mira.carey@mongodb.com</customfieldvalue>
            <customfieldvalue>akira.kurogane@gmail.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hudekn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hu3jo7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="3837">Sharding 2020-04-20</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;Use a 2+ shard cluster and 2 mongos nodes not being used by any other clients.&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Create a collection and shard on field &quot;a&quot;. No need to make it particularly big.&lt;br/&gt;
(To reproduce the issue for collStats I think the chunks need to be all on one shard. But if it has already split and balanced the dataSize command&apos;s issue will still be reproducible.)&lt;/li&gt;
	&lt;li&gt;Confirm that dataSize and collStats commands work correctly via both mongos nodes.&lt;/li&gt;
	&lt;li&gt;Using mongos #1 node drop the collection. Recreate and reshard by another field &quot;b&quot;. Split and balance it so chunks are on 2 or more shards.&lt;/li&gt;
	&lt;li&gt;On mongos #2 run dataSize and collStats commands without doing anything else with the collection first.&#160;
	&lt;ul&gt;
		&lt;li&gt;dataSize will fail with &quot;keyPattern must equal shard key&quot; error&lt;/li&gt;
		&lt;li&gt;collStats will only fetch from shard(s) that it used originally.&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;Run a single insert command on the collection. The dataSize, and I think also the collStats command, works correctly immediately after that.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;&#160;&lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hud0tz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>