<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:34:36 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-33797] sharding metadata command should make sure that ShardRegistry is up to date</title>
                <link>https://jira.mongodb.org/browse/SERVER-33797</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;... before accessing it.&lt;/p&gt;

&lt;p&gt;For example, shardCollection makes sure that there are shards to put the new sharded collection. It is possible that the shard already exists in storage but not yet loaded to the ShardRegistry, so it can mistakenly think that no shard exists.&lt;/p&gt;</description>
                <environment></environment>
        <key id="508986">SERVER-33797</key>
            <summary>sharding metadata command should make sure that ShardRegistry is up to date</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="matthew.saltz@mongodb.com">Matthew Saltz</assignee>
                                    <reporter username="randolph@mongodb.com">Randolph Tan</reporter>
                        <labels>
                    </labels>
                <created>Fri, 9 Mar 2018 22:50:41 +0000</created>
                <updated>Sun, 29 Oct 2023 22:33:55 +0000</updated>
                            <resolved>Thu, 29 Mar 2018 17:37:46 +0000</resolved>
                                    <version>3.7.2</version>
                                    <fixVersion>3.7.4</fixVersion>
                                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="1849337" author="xgen-internal-githook" created="Thu, 29 Mar 2018 17:30:51 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;email&apos;: &apos;matthew.saltz@mongodb.com&apos;, &apos;name&apos;: &apos;Matthew Saltz&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-33797&quot; title=&quot;sharding metadata command should make sure that ShardRegistry is up to date&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-33797&quot;&gt;&lt;del&gt;SERVER-33797&lt;/del&gt;&lt;/a&gt;: Reload ShardRegistry on shard collection command&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/6306b96749d4a547a1fa023eb1a0085b37e75c07&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/6306b96749d4a547a1fa023eb1a0085b37e75c07&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1846747" author="matthew.saltz" created="Tue, 27 Mar 2018 17:35:28 +0000"  >&lt;p&gt;Code review: &lt;a href=&quot;https://mongodbcr.appspot.com/190780002/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://mongodbcr.appspot.com/190780002/&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1843356" author="matthew.saltz" created="Fri, 23 Mar 2018 15:40:23 +0000"  >&lt;p&gt;Okay, that makes sense. So basically, it doesn&apos;t matter that the caller does the reload, it just matters that the reload has happened since the caller started its operation. If some other update sneaks in afterwards it doesn&apos;t matter because we just need to be sure we&apos;re seeing what we were supposed to see when we got called. (&apos;we&apos; being the caller).&lt;/p&gt;</comment>
                            <comment id="1843322" author="renctan" created="Fri, 23 Mar 2018 15:25:05 +0000"  >&lt;p&gt;When ShardRegistry::reload returns false, it just means that the current thread did not perform the reload. This is a scenario when some other thread was already reloading and to avoid repeated work, it just piggybacked on the work of another thread. Here is an example of why reload needs to be only called one extra time and not in a loop:&lt;/p&gt;

&lt;p&gt;Scenario:&lt;br/&gt;
1. Thread1 starts reload @ t1&lt;br/&gt;
2. config.shards was updated @ t2&lt;br/&gt;
3. Thread2 wishes to reload and the intention was to see the changes in t2.&lt;br/&gt;
4. Thread1 is already reloading, so Thread2 just waits for it to finish (and reap the rewards of Thread1&apos;s hard work).&lt;br/&gt;
5. Thread2 will get a false from calling reload, inspects the registry, but still couldn&apos;t find something it was expecting, this is because the reload started @ t1 and can miss the update @ t2 completely&lt;br/&gt;
6. Thread2 can start the reload again. And even if it returned false it can be sure that the 2nd round of reload will surely contain updates that happened at least in t2, so it doesn&apos;t need to reload again.&lt;/p&gt;

&lt;p&gt;enableSharding does call &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.7.3/src/mongo/db/s/config/sharding_catalog_manager_shard_operations.cpp#L892&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;reload&lt;/a&gt; when selecting shards. The issue in the build failure is that path is not taken because the old primary has already created the database entry so the new primary just &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.7.3/src/mongo/db/s/config/sharding_catalog_manager_database_operations.cpp#L104&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;skips the reload completely&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="1842777" author="matthew.saltz" created="Fri, 23 Mar 2018 00:03:43 +0000"  >&lt;p&gt;It&apos;s also possible I&apos;m looking in the entirely wrong places, in which case please tell me where I should be looking instead &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/smile.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/p&gt;</comment>
                            <comment id="1842776" author="matthew.saltz" created="Fri, 23 Mar 2018 00:03:06 +0000"  >&lt;p&gt;It seems like every call to &lt;tt&gt;getShard()&lt;/tt&gt; on the &lt;tt&gt;ShardRegistry&lt;/tt&gt; (which is what &lt;tt&gt;ShardingCatalogManager::shardCollection&lt;/tt&gt; and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/4ca974b2165d74c3a9bdc4a765d445f23aaa2664/src/mongo/s/client/shard_registry.cpp#L118&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;does attempt a reload:&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;However, from looking at the &lt;tt&gt;reload()&lt;/tt&gt; &lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/s/client/shard_registry.cpp#L270&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;implementation&lt;/a&gt;, despite what the comment says in the header file (that you should call it &quot;one more time&quot; if it returns false), it seems to me that it&apos;s possible that several repeated calls to reload could be necessary as long as another thread keeps interfering and doing the reload itself.&lt;/p&gt;

&lt;p&gt;I might be misunderstanding since there&apos;s clearly a lot of context I&apos;m missing but it seems like the two calls to &lt;tt&gt;reload()&lt;/tt&gt; inside &lt;tt&gt;getShard()&lt;/tt&gt; should be replaced with something along the lines of:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;while&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; (!shardRegistry-&amp;gt;reload()) {}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;(You could do it in another way for style reasons, but this is the gist.) If I&apos;m correct in my understanding, it could also be worth adding either an additional function or additional parameter to reload to specify whether the caller needs to truly perform the reload or if another thread doing the reload is sufficient. Also, I&apos;m curious to explore a bit more to see if there are situations where we&apos;re actually okay with the reload being done by some other thread or if in most cases we need to call it repeatedly until it returns true anyways.&lt;/p&gt;

&lt;p&gt;For &lt;tt&gt;ShardingCatalogManager::enableSharding&lt;/tt&gt;, on the other hand, the only time it touches the &lt;tt&gt;ShardRegistry&lt;/tt&gt; is to do &lt;tt&gt;createDatabase&lt;/tt&gt; when it goes to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/4ca974b2165d74c3a9bdc4a765d445f23aaa2664/src/mongo/db/s/config/sharding_catalog_manager_shard_operations.cpp#L886&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;pick a primary shard id for the new database&lt;/a&gt;, in which case &lt;tt&gt;ShardRegistry::getAllShardIds()&lt;/tt&gt; is called instead, which doesn&apos;t do any reloading internally. So then the reload only happens in the caller (&lt;tt&gt;_selectShardForNewDatabase()&lt;/tt&gt;) if &lt;tt&gt;getAllShardIds()&lt;/tt&gt; comes back empty. Is it possible that there&apos;s a case where all shard IDs does not come back empty but where the &lt;tt&gt;ShardRegistry&lt;/tt&gt; is out of date and should be reloaded anyways? (Is that what this JIRA is really referring to?) Also in this particular case the call-reload-till-it-returns-true protocol isn&apos;t followed - is that okay?&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=janna.golden&quot; class=&quot;user-hover&quot; rel=&quot;janna.golden&quot;&gt;janna.golden&lt;/a&gt; Tagging you since this is related to your comment in the build failure associated with this.&lt;/p&gt;
</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="550002">SERVER-35252</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 23 Mar 2018 00:03:06 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 45 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 45 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>15.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>matthew.saltz@mongodb.com</customfieldvalue>
            <customfieldvalue>randolph@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hts9kf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htkj4v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2216">Sharding 2018-04-09</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htrvrb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>