<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:12:36 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-46845] Shard which received a StaleShardVersion can get stuck indefinitely in a moveChunk command</title>
                <link>https://jira.mongodb.org/browse/SERVER-46845</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;When a shard updates its knowledge of its shard version, post migration commit, it logs a message which looks like this:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary] 2020-02-04T15:18:15.510+0000 I  COMMAND  [conn291] command admin.$cmd appName: &quot;tid:54&quot; command: getMore { getMore: 3975999160804323048, collection: &quot;$cmd.aggregate&quot;, lsid: { id: UUID(&quot;d2eb0b2e-2ff8-4263-ab12-e5f9514ff6a4&quot;), uid: BinData(0, E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855) }, $clusterTime: { clusterTime: Timestamp(1580829495, 39), signat[ShardedClusterFixture:job0:shard1:primary] 2020-02-04T15:18:11.560+0000 I  SHARDING [conn55] Updating metadata for collection config.system.sessions from collection version: 15|0||5e398add924cca4d6c4487b2, shard version: 0|0||5e398add924cca4d6c4487b2 to collection version: 16|0||5e398add924cca4d6c4487b2, shard version: 16|0||5e398add924cca4d6c4487b2 due to version change&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;This log line comes from &lt;a href=&quot;https://github.com/mongodb/mongo/commit/a84c09a19720b73cedb2e8ef7c5cfeedfa1c9761#diff-994a130a1b71b471427ee9d7208e5996L175&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt; and if we zoom inside &lt;a href=&quot;https://github.com/mongodb/mongo/blob/ed0c0365b4baf8939d39edb766c8ddbbc33187d3/src/mongo/db/s/collection_metadata.cpp#L83&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;CollectionMetadata::toStringBasic()&lt;/a&gt;, the call to log the current shard version will invoke &lt;a href=&quot;https://github.com/mongodb/mongo/blob/ed0c0365b4baf8939d39edb766c8ddbbc33187d3/src/mongo/s/chunk_manager.cpp#L432&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;ChunkManager::getVersion(ShardId)&lt;/tt&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;If it so happens that the CatalogCache&apos;s entry for a collection gets invalidated with the local shard id &lt;b&gt;and&lt;/b&gt; there is a concurrently running migration, it is possible that the completion of the chunk migration will get stuck indefinitely, because &lt;a href=&quot;https://github.com/mongodb/mongo/blob/ed0c0365b4baf8939d39edb766c8ddbbc33187d3/src/mongo/s/chunk_manager.cpp#L432&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;ChunkManager::getVersion&lt;/tt&gt;&lt;/a&gt; will keep throwing &lt;tt&gt;ShardInvalidatedForTargetingInfo&lt;/tt&gt; exceptions and will keep getting retried under &lt;a href=&quot;https://github.com/mongodb/mongo/blob/ed0c0365b4baf8939d39edb766c8ddbbc33187d3/src/mongo/db/s/migration_util.cpp#L722&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;refreshFilteringMetadataUntilSuccess&lt;/tt&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I think the bug is currently not happening, because somehow after the logging changes were &lt;a href=&quot;https://github.com/mongodb/mongo/commit/a84c09a19720b73cedb2e8ef7c5cfeedfa1c9761#diff-994a130a1b71b471427ee9d7208e5996R180&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;committed&lt;/a&gt; this line is no longer logged in the test output. For example &lt;a href=&quot;https://logkeeper.mongodb.org/lobster/build/f163351725e93b4a47592b0f52d53690/test/5e541d1fc2ab6876a6ed9bc1#bookmarks=0%2C1465&amp;amp;l=1&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1271620">SERVER-46845</key>
            <summary>Shard which received a StaleShardVersion can get stuck indefinitely in a moveChunk command</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="blake.oler@mongodb.com">Blake Oler</assignee>
                                    <reporter username="kaloian.manassiev@mongodb.com">Kaloian Manassiev</reporter>
                        <labels>
                            <label>sharding-4.4-stabilization</label>
                    </labels>
                <created>Fri, 13 Mar 2020 12:55:25 +0000</created>
                <updated>Sun, 29 Oct 2023 22:10:50 +0000</updated>
                            <resolved>Wed, 8 Apr 2020 14:23:14 +0000</resolved>
                                                    <fixVersion>4.4.0-rc2</fixVersion>
                    <fixVersion>4.7.0</fixVersion>
                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="3037175" author="xgen-internal-githook" created="Mon, 13 Apr 2020 17:05:57 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Blake Oler&apos;, &apos;email&apos;: &apos;blake.oler@mongodb.com&apos;, &apos;username&apos;: &apos;BlakeIsBlake&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-46845&quot; title=&quot;Shard which received a StaleShardVersion can get stuck indefinitely in a moveChunk command&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-46845&quot;&gt;&lt;del&gt;SERVER-46845&lt;/del&gt;&lt;/a&gt; Allow logging operations to bypass ShardInvalidatedForTargeting exception when accessing shard versions&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 6c88549226e8f68f06b19d9e9a1e0b5f756494b0)&lt;br/&gt;
Branch: v4.4&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/2fdb36056997121e483ce758fc96400ad40bb24e&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/2fdb36056997121e483ce758fc96400ad40bb24e&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3028508" author="xgen-internal-githook" created="Mon, 6 Apr 2020 23:27:59 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Blake Oler&apos;, &apos;email&apos;: &apos;blake.oler@mongodb.com&apos;, &apos;username&apos;: &apos;BlakeIsBlake&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-46845&quot; title=&quot;Shard which received a StaleShardVersion can get stuck indefinitely in a moveChunk command&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-46845&quot;&gt;&lt;del&gt;SERVER-46845&lt;/del&gt;&lt;/a&gt; Allow logging operations to bypass ShardInvalidatedForTargeting exception when accessing shard versions&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/6c88549226e8f68f06b19d9e9a1e0b5f756494b0&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/6c88549226e8f68f06b19d9e9a1e0b5f756494b0&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3021828" author="renctan" created="Wed, 1 Apr 2020 20:03:02 +0000"  >&lt;p&gt;Approach sounds good to me&lt;/p&gt;</comment>
                            <comment id="3020951" author="kaloian.manassiev" created="Wed, 1 Apr 2020 13:48:51 +0000"  >&lt;p&gt;Yes, placing the second catalog cache behind a feature flag, as opposed to &lt;tt&gt;testCommandsEnabled&lt;/tt&gt; seems like the right way to go (this is what is being discussed under BF-16423 and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-46726&quot; title=&quot;Provide separate setParameter to disable diagnostic work&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-46726&quot;&gt;&lt;del&gt;SERVER-46726&lt;/del&gt;&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;CC &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tommaso.tocci&quot; class=&quot;user-hover&quot; rel=&quot;tommaso.tocci&quot;&gt;tommaso.tocci&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3020059" author="blake.oler" created="Tue, 31 Mar 2020 20:48:34 +0000"  >&lt;p&gt;Confirmed my above guess via local testing. The addition of the second catalog cache obfuscates the underlying error. This is because the second catalog cache will never see shards marked as stale. This makes me nervous for our test coverage. I&apos;d like to be able to place the second catalog cache behind a feature flag, so that we&apos;re able to turn it on/off for different testing variants. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev&quot;&gt;kaloian.manassiev&lt;/a&gt; thoughts?&lt;/p&gt;

&lt;p&gt;As for the error itself, this is because we&apos;re attempting to access an outdated &lt;tt&gt;RoutingTableHistory&lt;/tt&gt; object to log its shard version. We should be able to access all versions for logging purposes without throwing a &lt;tt&gt;ShardInvalidatedForTargeting&lt;/tt&gt; exception. In a local patch, I&apos;ve added secondary &lt;tt&gt;getVersionForLogging&lt;/tt&gt; function calls that don&apos;t check if the version is stale, that we use if we&apos;re only trying to log. This fixes the issue. Good on the approach? &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=renctan&quot; class=&quot;user-hover&quot; rel=&quot;renctan&quot;&gt;renctan&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;But I&apos;m unable to test this issue via a JS test unless we turn off the second catalog cache via feature flag. I&apos;ll dig to see if I can test this via unit test, but this seems to be a scenario better tested with an integration test.&lt;/p&gt;</comment>
                            <comment id="2979211" author="blake.oler" created="Fri, 13 Mar 2020 18:23:00 +0000"  >&lt;p&gt;The most educated guess I can make at the moment is not that the logging changes affected the behavior to allow the affected BF to pass again. Rather, it&apos;s that the introduction of the second catalog cache (introduced around the same time) allowed this function to pass. I don&apos;t know if this is desired behavior or not yet. I will need to spend time thinking about how the second cache interacts with PM-1633. For now, I wouldn&apos;t say this is a 4.4-rc0 blocking issue.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="18953"><![CDATA[v4.4]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 13 Mar 2020 18:23:00 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 43 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 43 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>15.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>blake.oler@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
            <customfieldvalue>randolph@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hx825r:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hvdvi7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="3744">Sharding 2020-03-23</customfieldvalue>
    <customfieldvalue id="3745">Sharding 2020-04-06</customfieldvalue>
    <customfieldvalue id="3837">Sharding 2020-04-20</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hx7of3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>