<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:30:17 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-32448] ShardRegistry::reload() does blocking work on NetworkInterface thread</title>
                <link>https://jira.mongodb.org/browse/SERVER-32448</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;The ShardRegistry schedules itself to perform _internalReload()s periodically, using the TaskExecutor.  These jobs are dispatched to NetworkInterfaceASIO, and will eventually run on its thread.&lt;/p&gt;

&lt;p&gt;From _internalReload(), we call reload(), which tries to getAllShards() from the ShardingCatalogClientImpl.  getAllShards() makes a Fetcher instance, which launches networking work, and then waits for it to join().  However, when we run this, we are already on NetworkInterfaceASIO&apos;s thread.  This breaks the contract that callbacks to NetworkInterfaceASIO may not perform blocking work.  Worse, when these calls are issued through the same TaskExecutor, the thread will deadlock.&lt;/p&gt;</description>
                <environment></environment>
        <key id="475404">SERVER-32448</key>
            <summary>ShardRegistry::reload() does blocking work on NetworkInterface thread</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13202">Works as Designed</resolution>
                                        <assignee username="backlog-server-sharding">[DO NOT USE] Backlog - Sharding Team</assignee>
                                    <reporter username="samantha.ritter@mongodb.com">Samantha Ritter</reporter>
                        <labels>
                            <label>ASE</label>
                            <label>ASIO</label>
                            <label>Sharding</label>
                    </labels>
                <created>Thu, 21 Dec 2017 20:38:17 +0000</created>
                <updated>Fri, 27 Oct 2023 13:54:02 +0000</updated>
                            <resolved>Fri, 4 May 2018 14:20:31 +0000</resolved>
                                    <version>3.4.14</version>
                    <version>3.6.4</version>
                                                    <component>Networking</component>
                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="1882918" author="kaloian.manassiev" created="Fri, 4 May 2018 14:20:31 +0000"  >&lt;p&gt;The blocking work is done on a completely isolated TaskExecutor, which should not interfere with any other executors, which are used for a real networking work. Given the changes that the Platforms team is doing for ASIO, I am opting to close this ticket as WAD.&lt;/p&gt;</comment>
                            <comment id="1759020" author="schwerin" created="Sun, 24 Dec 2017 14:34:12 +0000"  >&lt;p&gt;Take a look at the AsyncResultsMerger. It has a similar task, and a more modern API. May not be directly useful, but worth understanding.&lt;/p&gt;</comment>
                            <comment id="1758314" author="samantha.ritter@10gen.com" created="Thu, 21 Dec 2017 22:45:42 +0000"  >&lt;p&gt;After talking to &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=misha.tyulenev&quot; class=&quot;user-hover&quot; rel=&quot;misha.tyulenev&quot;&gt;misha.tyulenev&lt;/a&gt; about this a little, some callers of ShardRegistry::reload() want to use it synchronously to run manual refreshes.  It seems like the best way to approach this would be to offer a synchronous reload() (the existing one) as well as an asynchronous one, to run the internal periodic refreshes, which can be async.  To write a fully async reload() method requires an async _exhaustiveFindOnConfig, which requires modifications to the Fetcher so that it can be callback-driven instead of notifying waiters when it is finished.  This shouldn&apos;t be too difficult to do, though, it&apos;s just a matter of re-framing the existing code around callbacks.&lt;/p&gt;</comment>
                            <comment id="1758238" author="samantha.ritter@10gen.com" created="Thu, 21 Dec 2017 21:28:16 +0000"  >&lt;p&gt;It&apos;s true that the commenting in task_executor.h for the scheduleWork() method doesn&apos;t say anything about what kind of work may or may not be scheduled.  However, all the work scheduled through the task executor (networking or not) is going to get run through asio&apos;s async execution engine.  Any blocking work that runs in a callback will clog up the works.&lt;/p&gt;

&lt;p&gt;I am doing a POC to see if we can have TaskExecutors share NetworkInterfaceASIO instances, so that they can all share the same connection pool.  This forces all the users of the TaskExecutors to share one asio thread, which is how I found this now.  Before, you&apos;re right, the networking call to _exhaustiveFindOnConfig was going through a different TaskExecutor, which would have had its own NetworkInterfaceASIO instance with its own thread, so it wouldn&apos;t have deadlocked.&lt;/p&gt;

&lt;p&gt;Ok, so I think there are two main ways to address this that would unblock my POC for the Egress Deployment project:&lt;br/&gt;
1. Offer an asynchronous _exhaustiveFindOnConfig method that this path could use instead.  The problem is that this mostly-asynchronous workflow tries to do synchronous work (inside _exhaustiveFindOnConfig).&lt;br/&gt;
2. Do not use the TaskExecutor for this periodic scheduling, maybe the PeriodicRunner could be used instead?  There is no 1:1 alternative to the scheduleWork() method of running future tasks that I know of, but maybe it can just run through the PeriodicRunner the way some other regularly-occurring background jobs do in the codebase.&lt;/p&gt;</comment>
                            <comment id="1758218" author="kaloian.manassiev" created="Thu, 21 Dec 2017 21:10:35 +0000"  >&lt;p&gt;I believe the TaskExecutor backed by ASIO was the only option available to schedule tasks at some point in the future. Is there an alternative to that?&lt;/p&gt;

&lt;p&gt;Also just for the record, the TaskExecutor which is used for reload should be a completely separate executor and shouldn&apos;t be used for networking I/O. Did you find this to not be the case?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="471221">SERVER-32270</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25141"><![CDATA[Sharding]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 21 Dec 2017 21:10:35 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 40 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 40 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-sharding</customfieldvalue>
            <customfieldvalue>schwerin@mongodb.com</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
            <customfieldvalue>samantha.ritter@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htmr6f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr9ed3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2050">Platforms 2018-01-01</customfieldvalue>
    <customfieldvalue id="2092">Platforms 2018-01-15</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htmdav:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>