<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:29:52 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-53084] ServerDiscoveryMonitor::shutdown can lead to deadlock</title>
                <link>https://jira.mongodb.org/browse/SERVER-53084</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;In a callback for TaskExecutor::scheduleRemoteCommand &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a194505325087b1e841fdee55c51312a042ce9d2/src/mongo/client/server_discovery_monitor.cpp#L319&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;, we try to acquire a mutex. &lt;br/&gt;
 This is a problem because in &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a194505325087b1e841fdee55c51312a042ce9d2/src/mongo/client/server_discovery_monitor.cpp#L364-L376&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;ServerDiscoveryMonitor::shutdown, we acquire the mutex before canceling the remaining tasks in the executor and shutting down the executor&lt;/a&gt;, and either cancellation or shutdown+join can cause the callbacks to run, leading to a deadlock. &lt;/p&gt;

&lt;p&gt;We should either reset the executor pointer and cancel the handles outside of the lock, or check the status passed into the callback for scheduleRemoteCommand and only acquire the mutex if it&apos;s an OK status.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1552325">SERVER-53084</key>
            <summary>ServerDiscoveryMonitor::shutdown can lead to deadlock</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="matthew.saltz@mongodb.com">Matthew Saltz</assignee>
                                    <reporter username="matthew.saltz@mongodb.com">Matthew Saltz</reporter>
                        <labels>
                            <label>servicearch-wfbf-day</label>
                    </labels>
                <created>Wed, 25 Nov 2020 21:55:48 +0000</created>
                <updated>Sun, 29 Oct 2023 22:00:00 +0000</updated>
                            <resolved>Tue, 22 Dec 2020 18:08:51 +0000</resolved>
                                                    <fixVersion>4.9.0</fixVersion>
                                    <component>Networking</component>
                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="3547918" author="ian@10gen.com" created="Mon, 4 Jan 2021 22:01:29 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;username&apos;: u&apos;evrg-bot-webhook&apos;, &apos;name&apos;: u&apos;Matthew Saltz&apos;, &apos;email&apos;: u&apos;matthew.saltz@mongodb.com&apos;}
&lt;p&gt;Message:&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-53084&quot; title=&quot;ServerDiscoveryMonitor::shutdown can lead to deadlock&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-53084&quot;&gt;&lt;del&gt;SERVER-53084&lt;/del&gt;&lt;/a&gt; Run onFinish callback out of line in the NetworkInterfaceTL&lt;br/&gt;
Branch:master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/e237f80590fe661eed0e3c9ea7988b8306dc3604&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/e237f80590fe661eed0e3c9ea7988b8306dc3604&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3540213" author="matthew.saltz" created="Tue, 22 Dec 2020 21:07:42 +0000"  >&lt;p&gt;Gotcha, thought that might have been what happened &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/smile.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt; I was just clarifying because we went with a solution in the NetworkInterfaceTL and not the ServerDiscoveryMonitor.&lt;/p&gt;</comment>
                            <comment id="3540116" author="daniel.gottlieb@10gen.com" created="Tue, 22 Dec 2020 20:08:25 +0000"  >&lt;p&gt;Ah, that makes sense then! The lack of a github push notification led me to believe that the CR was closed without pushing because a different problem was discovered.&lt;/p&gt;</comment>
                            <comment id="3540101" author="matthew.saltz" created="Tue, 22 Dec 2020 19:54:35 +0000"  >&lt;p&gt;Not that I&apos;m aware of... Why do you ask?&lt;/p&gt;

&lt;p&gt;Also just to clarify, the linked CR did get pushed. For whatever reason the &lt;a href=&quot;https://github.com/mongodb/mongo/commit/e237f80590fe661eed0e3c9ea7988b8306dc3604&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;github link&lt;/a&gt; didn&apos;t get added to the ticket automatically.&lt;/p&gt;</comment>
                            <comment id="3539926" author="daniel.gottlieb@10gen.com" created="Tue, 22 Dec 2020 18:12:41 +0000"  >&lt;p&gt;Apologies for commenting from the sideline: were the linked testing failures fixed (perhaps accidentally) as part of a different patch?&lt;/p&gt;</comment>
                            <comment id="3539915" author="matthew.saltz" created="Tue, 22 Dec 2020 18:08:51 +0000"  >&lt;p&gt;Closing this ticket since ServerDiscoveryMonitor was actually correct given the TaskExecutor::cancel contract. We can make a new ticket if we want to change the locking around for cleanup purposes.&lt;/p&gt;</comment>
                            <comment id="3533394" author="JIRAUSER1256988" created="Wed, 16 Dec 2020 21:10:10 +0000"  >&lt;p&gt;The&#160;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/6ee6a9d6f6ff15fc65bf13baeab3717dfb72eb20/src/mongo/executor/thread_pool_task_executor.cpp#L480-L483&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;ThreadPoolTaskExecutor::cancel&lt;/a&gt; has its own lock and lock nesting while calling it from&#160;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/a194505325087b1e841fdee55c51312a042ce9d2/src/mongo/client/server_discovery_monitor.cpp#L364-L376&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;SingleServerDiscoveryMonitor::shutdown()&lt;/a&gt;&#160;and then from&#160; &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a194505325087b1e841fdee55c51312a042ce9d2/src/mongo/client/server_discovery_monitor.cpp#L389-L395&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;SingleServerDiscoveryMonitor::_cancelOutstandingRequest&lt;/a&gt;&#160;is protecting the fields like `_remoteCommandHandle`, `_executor`, but is not necessary to hold the mutex while invoking the `_executor-&amp;gt;cancel()`. I think this should be refactored. Will it be enough to fix the deadlock?&lt;/p&gt;

&lt;p&gt;Save the _executor in local variable and invoke cancel() outside of the lock, removing a nested lock unless absolutely necessary should always be a nice cleanup.&lt;/p&gt;</comment>
                            <comment id="3533282" author="matthew.saltz" created="Wed, 16 Dec 2020 20:29:53 +0000"  >&lt;p&gt;I now think that this actually started happening after/due to &lt;a href=&quot;https://github.com/mongodb/mongo/commit/c50282b539308d1578465fa0819e228a47d3421e&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this commit to make the RSM call drop() in its destructor&lt;/a&gt;. FYI &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=andrew.shuvalov&quot; class=&quot;user-hover&quot; rel=&quot;andrew.shuvalov&quot;&gt;andrew.shuvalov&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="3533259" author="matthew.saltz" created="Wed, 16 Dec 2020 20:18:00 +0000"  >&lt;p&gt;The problem is that the RSM obtained &lt;a href=&quot;https://github.com/mongodb/mongo/blob/04401a326046babcb390cc444f52699f732b3b2f/src/mongo/client/replica_set_monitor_manager.cpp#L79&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt; can be the last shared_ptr to that RSM, leading to it being destroyed at the end of this scope. This is a problem because validateHosts is happening in a networking thread, which leads the cancellation logic to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/704e695b7296e52bf47d999b8250a14809752051/src/mongo/executor/network_interface_thread_pool.cpp#L132-L134&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;run inline&lt;/a&gt;, leading to the double-mutex acquisition&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="1585747">SERVER-53684</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="2356256">SERVER-77707</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 16 Dec 2020 21:10:10 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 5 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 5 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>38.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>andrew.shuvalov@mongodb.com</customfieldvalue>
            <customfieldvalue>daniel.gottlieb@mongodb.com</customfieldvalue>
            <customfieldvalue>ian@mongodb.com</customfieldvalue>
            <customfieldvalue>matthew.saltz@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hyj453:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hy784f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4274">Service arch 2020-12-28</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hyiqef:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>