<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:20:06 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-49508] Step up deadlock between migration recovery and prepared transaction</title>
                <link>https://jira.mongodb.org/browse/SERVER-49508</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;As part of stepping up to primary, a node &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d3430cddfc340f76ac71a58587a1a14c6c506b05/src/mongo/db/repl/replication_coordinator_external_state_impl.cpp#L896&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;recovers migrations that were active on the previous primary&lt;/a&gt;. This involves &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d3430cddfc340f76ac71a58587a1a14c6c506b05/src/mongo/db/s/migration_util.cpp#L793-L796&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;synchronously scanning every document in &lt;tt&gt;config.migrationCoordinators&lt;/tt&gt;&lt;/a&gt; and clearing the metadata for each corresponding namespace, which requires taking a &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d3430cddfc340f76ac71a58587a1a14c6c506b05/src/mongo/db/s/migration_util.cpp#L808&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;collection X lock&lt;/a&gt;. This happens after the primary restores the locks of active prepared transactions (through &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d3430cddfc340f76ac71a58587a1a14c6c506b05/src/mongo/db/session_catalog_mongod.cpp#L227-L231&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;MongoDSessionCatalog::onStepUp()&lt;/a&gt; called &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c02d19aebdba13451d576f56885e3358dad90e48/src/mongo/db/repl/replication_coordinator_external_state_impl.cpp#L478&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;), so if a prepared transaction has a lock on a namespace that had an active migration, taking the collection X lock on that namespace in migration recovery will block, preventing the step up from succeeding.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1409308">SERVER-49508</key>
            <summary>Step up deadlock between migration recovery and prepared transaction</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="tommaso.tocci@mongodb.com">Tommaso Tocci</assignee>
                                    <reporter username="jack.mulrow@mongodb.com">Jack Mulrow</reporter>
                        <labels>
                    </labels>
                <created>Tue, 14 Jul 2020 21:32:42 +0000</created>
                <updated>Sun, 29 Oct 2023 22:05:49 +0000</updated>
                            <resolved>Fri, 24 Jul 2020 09:10:47 +0000</resolved>
                                                    <fixVersion>4.7.0</fixVersion>
                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="3300120" author="xgen-internal-githook" created="Fri, 24 Jul 2020 09:07:58 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Tommaso Tocci&apos;, &apos;email&apos;: &apos;tommaso.tocci@mongodb.com&apos;, &apos;username&apos;: &apos;toto-dev&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-49508&quot; title=&quot;Step up deadlock between migration recovery and prepared transaction&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-49508&quot;&gt;&lt;del&gt;SERVER-49508&lt;/del&gt;&lt;/a&gt; Step up deadlock between migration recovery and prepared transaction&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/c857e1dcb22bb56af4302a9f0997106996ad1f3f&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/c857e1dcb22bb56af4302a9f0997106996ad1f3f&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3294473" author="jack.mulrow" created="Tue, 21 Jul 2020 14:35:17 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev&quot;&gt;kaloian.manassiev&lt;/a&gt;, yeah the problem is the step up thread taking that X lock. Getting rid of the acquisition should fix this or we could possibly move migration recovery before step up restores the locks for prepared transactions, but I&apos;m not sure what other problems might fall out of that.&lt;/p&gt;</comment>
                            <comment id="3293992" author="kaloian.manassiev" created="Tue, 21 Jul 2020 08:59:17 +0000"  >&lt;p&gt;Just to be clear &amp;#8211; the issue is the X-lock which is being taken to clear the filtering info, right? Because I don&apos;t think we need to take it for clearing, it must be a vestige.&lt;/p&gt;</comment>
                            <comment id="3293560" author="esha.maharishi@10gen.com" created="Mon, 20 Jul 2020 23:15:54 +0000"  >&lt;p&gt;Sounds good, reassigning to &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tommaso.tocci&quot; class=&quot;user-hover&quot; rel=&quot;tommaso.tocci&quot;&gt;tommaso.tocci&lt;/a&gt;. Feel free to re-triage as you think makes sense.&lt;/p&gt;</comment>
                            <comment id="3293527" author="jack.mulrow" created="Mon, 20 Jul 2020 22:48:41 +0000"  >&lt;p&gt;This particular issue should only affect master since the step up thread in 4.4 triggers the entire migration recovery to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/97cd266f1b5df7f3af81d487e6dab2ca1060935a/src/mongo/db/s/migration_util.cpp#L838&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;run on the migration executor&lt;/a&gt; whereas in master the step up thread &lt;a href=&quot;https://github.com/mongodb/mongo/blob/92bab3bd50160122b7b5ecaa4126c43fc6bab6d6/src/mongo/db/s/migration_util.cpp#L807-L814&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;takes a collection lock itself before running the rest of recovery on the migration executor&lt;/a&gt;. I also can&apos;t reproduce the hang on 4.4 with the test I attached.&lt;/p&gt;</comment>
                            <comment id="3291935" author="esha.maharishi@10gen.com" created="Mon, 20 Jul 2020 13:51:02 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jack.mulrow&quot; class=&quot;user-hover&quot; rel=&quot;jack.mulrow&quot;&gt;jack.mulrow&lt;/a&gt;, could you confirm this issue only affects master, not 4.4?&lt;/p&gt;

&lt;p&gt;On 4.4, I think migration recovery only acquires a &lt;a href=&quot;https://github.com/mongodb/mongo/blob/97cd266f1b5df7f3af81d487e6dab2ca1060935a/src/mongo/db/s/migration_util.cpp#L919&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;collection IS lock&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;If so, this doesn&apos;t need to block 4.4 GA and I&apos;ll pass this over to &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tommaso.tocci&quot; class=&quot;user-hover&quot; rel=&quot;tommaso.tocci&quot;&gt;tommaso.tocci&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="269908" name="stepup_deadlock_repro.js" size="2967" author="jack.mulrow@mongodb.com" created="Tue, 14 Jul 2020 22:00:20 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 20 Jul 2020 13:51:02 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 28 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 28 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>35.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>esha.maharishi@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>jack.mulrow@mongodb.com</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
            <customfieldvalue>tommaso.tocci@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hxutbz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hxhodz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4135">Sharding 2020-07-27</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hxuflb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>