<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 08:11:27 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[DOCS-14892] Investigate changes in SERVER-59965: Distributed deadlock between renameCollection and multi-shard transaction</title>
                <link>https://jira.mongodb.org/browse/DOCS-14892</link>
                <project id="10380" key="DOCS">Documentation</project>
                    <description>    &lt;div class=&quot;panel&quot; style=&quot;background-color: #c2d2c2;border-width: 1px;&quot;&gt;&lt;div class=&quot;panelHeader&quot; style=&quot;border-bottom-width: 1px;background-color: #239eb0;&quot;&gt;&lt;b&gt;Downstream Change Summary&lt;/b&gt;&lt;/div&gt;&lt;div class=&quot;panelContent&quot; style=&quot;background-color: #c2d2c2;&quot;&gt;
&lt;p&gt;    Added new &apos;metadataRefreshInTransactionMaxWaitBehindCritSecMS&apos; server parameter.&lt;/p&gt;
&lt;/div&gt;&lt;/div&gt;

&lt;h2&gt;&lt;a name=&quot;DescriptionofLinkedTicket&quot;&gt;&lt;/a&gt;Description of Linked Ticket&lt;/h2&gt;
&lt;p&gt;    As part of a sharded renameCollection, the DDLCoordinator &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a444c63c396129836c6a5f230a465e9cc651e921/src/mongo/db/s/rename_collection_coordinator.cpp#L265-L297&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;instructs all participant shards&lt;/a&gt; to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a444c63c396129836c6a5f230a465e9cc651e921/src/mongo/db/s/rename_collection_participant_service.cpp#L250-L251&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;enter their critical sections&lt;/a&gt;. When all shards have entered it, the coordinator will do some work on the configsvr and finally it will tell the shards to leave their critical section.&lt;/p&gt;

&lt;p&gt;When running renameCollection concurrently with multi-shard transactions that affect that same collection, there exists a particular interleaving that can lead to a distributed deadlock:&lt;br/&gt;
1. shard0 receives the RenameCollectionParticipant command and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a444c63c396129836c6a5f230a465e9cc651e921/src/mongo/db/s/rename_collection_participant_service.cpp#L250-L251&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;enters its critical section&lt;/a&gt;&lt;br/&gt;
2. shard0 attempts to run an statement of the multi-shard txn. Since the critical section is taken, it will &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a444c63c396129836c6a5f230a465e9cc651e921/src/mongo/db/s/collection_sharding_runtime.cpp#L343-L349&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;throw StaleConfig&lt;/a&gt;. This error will &lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/db/service_entry_point_common.cpp#L1598-L1612&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;be caught on the way out of the command&lt;/a&gt; and it will &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a444c63c396129836c6a5f230a465e9cc651e921/src/mongo/db/service_entry_point_common.cpp#L1605&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;attempt to refresh the shardVersion&lt;/a&gt;. However, since the critical section is taken, the refresh will &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a444c63c396129836c6a5f230a465e9cc651e921/src/mongo/db/s/shard_filtering_metadata_refresh.cpp#L226&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;block until the critical section is released&lt;/a&gt;.&lt;br/&gt;
3. shard1 runs it&apos;s part of that multi-shard transaction, which will acquire the collection lock in MODE_IX, and then stash the locks.&lt;br/&gt;
4. shard1 receives the RenameCollectionParticipant and attempts to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a444c63c396129836c6a5f230a465e9cc651e921/src/mongo/db/s/rename_collection_participant_service.cpp#L250-L251&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;enter the critical section&lt;/a&gt;. However, since the transaction at point 3 had stashed the collection lock, we are not able to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a444c63c396129836c6a5f230a465e9cc651e921/src/mongo/db/s/recoverable_critical_section_service.cpp#L85&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;acquire the collection lock in MODE_S&lt;/a&gt; needed to enter the critical section.&lt;/p&gt;

&lt;p&gt;At this point we are deadlocked:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;shard0 is holding the critical section and won&apos;t release until shard1 acquires theirs.&lt;/li&gt;
	&lt;li&gt;shard1 Is holding the collection lock in MODE_IX until the txn gets committed, which won&apos;t happen because the txn (or perhaps, rather the refresh) is not making progress on shard0 due to the critical section.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;More generally, I believe this situation can occur in any DDL operation that needs to acquire the critical section in several nodes at the same time. I believe that resharding may also be affected by this.&lt;/p&gt;


</description>
                <environment></environment>
        <key id="1908080">DOCS-14892</key>
            <summary>Investigate changes in SERVER-59965: Distributed deadlock between renameCollection and multi-shard transaction</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="dave.cuthbert@mongodb.com">Dave Cuthbert</assignee>
                                    <reporter username="backlog-server-pm">Backlog - Core Eng Program Management Team</reporter>
                        <labels>
                    </labels>
                <created>Mon, 25 Oct 2021 07:59:27 +0000</created>
                <updated>Mon, 13 Nov 2023 17:47:55 +0000</updated>
                            <resolved>Wed, 23 Feb 2022 19:47:38 +0000</resolved>
                                                    <fixVersion>5.0.4</fixVersion>
                    <fixVersion>5.2.0</fixVersion>
                    <fixVersion>5.1.0-rc3</fixVersion>
                    <fixVersion>Server_Docs_20231030</fixVersion>
                    <fixVersion>Server_Docs_20231106</fixVersion>
                    <fixVersion>Server_Docs_20231105</fixVersion>
                    <fixVersion>Server_Docs_20231113</fixVersion>
                                    <component>manual</component>
                    <component>Server</component>
                        <due></due>
                            <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="4372521" author="xgen-internal-githook" created="Wed, 23 Feb 2022 19:53:19 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Dave&apos;, &apos;email&apos;: &apos;69165704+davemungo@users.noreply.github.com&apos;, &apos;username&apos;: &apos;davemungo&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/DOCS-14892&quot; title=&quot;Investigate changes in SERVER-59965: Distributed deadlock between renameCollection and multi-shard transaction&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DOCS-14892&quot;&gt;&lt;del&gt;DOCS-14892&lt;/del&gt;&lt;/a&gt; BACKPORT (#691)&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/DOCS-14892&quot; title=&quot;Investigate changes in SERVER-59965: Distributed deadlock between renameCollection and multi-shard transaction&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DOCS-14892&quot;&gt;&lt;del&gt;DOCS-14892&lt;/del&gt;&lt;/a&gt; BACKPORT&lt;/li&gt;
&lt;/ul&gt;


&lt;ul&gt;
	&lt;li&gt;Remove 5.2 release notes&lt;/li&gt;
&lt;/ul&gt;


&lt;ul&gt;
	&lt;li&gt;Remove 5.0 release notes&lt;br/&gt;
Branch: v5.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/10gen/docs-mongodb-internal/commit/36be09df8b48e06f588df78c038745dc7a9ae0b0&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/10gen/docs-mongodb-internal/commit/36be09df8b48e06f588df78c038745dc7a9ae0b0&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="4372375" author="xgen-internal-githook" created="Wed, 23 Feb 2022 19:07:35 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Dave&apos;, &apos;email&apos;: &apos;69165704+davemungo@users.noreply.github.com&apos;, &apos;username&apos;: &apos;davemungo&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/DOCS-14892&quot; title=&quot;Investigate changes in SERVER-59965: Distributed deadlock between renameCollection and multi-shard transaction&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DOCS-14892&quot;&gt;&lt;del&gt;DOCS-14892&lt;/del&gt;&lt;/a&gt; BACKPORT (#690)&lt;br/&gt;
Branch: v5.2&lt;br/&gt;
&lt;a href=&quot;https://github.com/10gen/docs-mongodb-internal/commit/a2b88baa7c4dd6facf545813756b66a34d8762a9&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/10gen/docs-mongodb-internal/commit/a2b88baa7c4dd6facf545813756b66a34d8762a9&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4372263" author="xgen-internal-githook" created="Wed, 23 Feb 2022 18:35:53 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Dave&apos;, &apos;email&apos;: &apos;69165704+davemungo@users.noreply.github.com&apos;, &apos;username&apos;: &apos;davemungo&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/DOCS-14892&quot; title=&quot;Investigate changes in SERVER-59965: Distributed deadlock between renameCollection and multi-shard transaction&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DOCS-14892&quot;&gt;&lt;del&gt;DOCS-14892&lt;/del&gt;&lt;/a&gt; distributed deadlock in transactions v5.3 (#678)&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/DOCS-14892&quot; title=&quot;Investigate changes in SERVER-59965: Distributed deadlock between renameCollection and multi-shard transaction&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DOCS-14892&quot;&gt;&lt;del&gt;DOCS-14892&lt;/del&gt;&lt;/a&gt; Distributed deadlock in transactions&lt;/li&gt;
&lt;/ul&gt;


&lt;ul&gt;
	&lt;li&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/DOCS-14892&quot; title=&quot;Investigate changes in SERVER-59965: Distributed deadlock between renameCollection and multi-shard transaction&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DOCS-14892&quot;&gt;&lt;del&gt;DOCS-14892&lt;/del&gt;&lt;/a&gt; Distributed deadlock in transactions&lt;/li&gt;
&lt;/ul&gt;


&lt;ul&gt;
	&lt;li&gt;Staging fixes&lt;/li&gt;
&lt;/ul&gt;


&lt;ul&gt;
	&lt;li&gt;Staging fixes&lt;/li&gt;
&lt;/ul&gt;


&lt;ul&gt;
	&lt;li&gt;Review feedback&lt;/li&gt;
&lt;/ul&gt;


&lt;ul&gt;
	&lt;li&gt;Staging fixes&lt;/li&gt;
&lt;/ul&gt;


&lt;ul&gt;
	&lt;li&gt;Review feedback&lt;/li&gt;
&lt;/ul&gt;


&lt;ul&gt;
	&lt;li&gt;Review feedback&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/10gen/docs-mongodb-internal/commit/14383e80542093fbfc62ce33daaa9d5fb3418b43&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/10gen/docs-mongodb-internal/commit/14383e80542093fbfc62ce33daaa9d5fb3418b43&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="4144620" author="dbeng-pm-bot" created="Mon, 25 Oct 2021 07:59:31 +0000"  >&lt;p&gt;&lt;em&gt;Downstream changes updated for upstream &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-59965&quot; title=&quot;Distributed deadlock between renameCollection and multi-shard transaction&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-59965&quot;&gt;&lt;del&gt;SERVER-59965&lt;/del&gt;&lt;/a&gt;:&lt;/em&gt; &lt;br/&gt;
Added new &apos;metadataRefreshInTransactionMaxWaitBehindCritSecMS&apos; server parameter.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10320">
                    <name>Documented</name>
                                            <outwardlinks description="documents">
                                        <issuelink>
            <issuekey id="1875034">SERVER-59965</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="1913006">DOCS-14907</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 25 Oct 2021 07:59:31 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 50 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>DOCSP-19447</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_14873" key="com.atlassian.jira.plugin.system.customfieldtypes:multiuserpicker">
                        <customfieldname>External Reviewer</customfieldname>
                        <customfieldvalues>
                                    <customfieldvalue><![CDATA[jordi.serra-torrens@mongodb.com]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>emet.ozar@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 50 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-pm</customfieldvalue>
            <customfieldvalue>dave.cuthbert@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>dbeng-pm-bot</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i07dyn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hzqyu7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0703z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                </customfields>
    </item>
</channel>
</rss>