<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:30:08 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-53168] Support 50 concurrent migrations on a single recipient</title>
                <link>https://jira.mongodb.org/browse/SERVER-53168</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Currently our tenant migration recipient thread pool default size is 8 (and it&#8217;s a tunable server startup parameter).  For each migration, we have  components, like  oplog fetcher &amp;amp; cloner, on recipient side that would do some synchronous job (fetching data from remote donor node) on the tenant migration recipient thread , without yielding the thread. With the default thread pool size as 8, we can  expect only at most 3 concurrent migration to be initiated on recipient side (per migration,  2 threads for sync jobs + 1 thread for async job),. Otherwise, concurrent tenant migration can lead to complete stalling of all active tenant migrations on recipient side. &lt;/p&gt;

&lt;p&gt;Consider the case, say,  tenant migration recipient thread pool size is 4.&lt;br/&gt;
1) Assume Recipient received recipeintSyncData comand  for migration id 1, 2,3 and all of them have started the oplog fetcher and at &lt;a href=&quot;https://github.com/mongodb/mongo/blob/90f767adbb1901d007ee4dd8714f53402d893669/src/mongo/db/repl/oplog_fetcher.cpp#L349&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;runQuery()&lt;/a&gt;. At this point, we are left with only one free worker thread  in the tenant migration recipient thread pool &lt;br/&gt;
2) Now, the recipient received recipeintSyncData comand  for migration id #4, that would successfully able to start the oplog fetcher&lt;/p&gt;

&lt;p&gt;So, now, we have no free worker threads left in the tenant migration recipient thread pool to  start the cloner. All 4 tenant migrations would hang on recipient side until we cancel one migration explicitly using ForgetMigration cmd.&lt;/p&gt;

</description>
                <environment></environment>
        <key id="1556347">SERVER-53168</key>
            <summary>Support 50 concurrent migrations on a single recipient</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="lingzhi.deng@mongodb.com">Lingzhi Deng</assignee>
                                    <reporter username="suganthi.mani@mongodb.com">Suganthi Mani</reporter>
                        <labels>
                            <label>pm-1791_milestone-B</label>
                    </labels>
                <created>Tue, 1 Dec 2020 20:32:29 +0000</created>
                <updated>Sun, 29 Oct 2023 21:59:51 +0000</updated>
                            <resolved>Wed, 17 Feb 2021 22:33:52 +0000</resolved>
                                                    <fixVersion>4.9.0</fixVersion>
                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="3620436" author="xgen-internal-githook" created="Wed, 17 Feb 2021 22:27:08 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Lingzhi Deng&apos;, &apos;email&apos;: &apos;lingzhi.deng@mongodb.com&apos;, &apos;username&apos;: &apos;ldennis&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-53168&quot; title=&quot;Support 50 concurrent migrations on a single recipient&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-53168&quot;&gt;&lt;del&gt;SERVER-53168&lt;/del&gt;&lt;/a&gt;: Set maxTenantMigrationDonorThreadPoolSize default to 128&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/8bf7eabbeeb393d30f6dfbbf941505a6d282de70&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/8bf7eabbeeb393d30f6dfbbf941505a6d282de70&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3600846" author="JIRAUSER1256988" created="Fri, 5 Feb 2021 02:57:44 +0000"  >&lt;p&gt;Added &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-54328&quot; title=&quot;Refactor creation of transient SSLConnectionContext to own its own instance of SSLManagerInterface&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-54328&quot;&gt;&lt;del&gt;SERVER-54328&lt;/del&gt;&lt;/a&gt; as dependency, this is the real problem, it became apparent only thanks to your stress test, it is actually pretty unfortunate that we understood the problem at the testing phase. Already started the refactoring.&lt;/p&gt;</comment>
                            <comment id="3536740" author="lingzhi.deng" created="Fri, 18 Dec 2020 20:15:27 +0000"  >&lt;p&gt;In our discussion with Cloud, we decided to support &amp;lt;= 50 concurrent migrations on a single recipient set (at least for private beta). And we will revisit this and refactor the OplogFetcher and the cloners code if needed. So I think we can change the thread pool to be 150 for now.&lt;/p&gt;</comment>
                            <comment id="3511311" author="suganthi.mani" created="Tue, 1 Dec 2020 21:06:19 +0000"  >&lt;p&gt;I think, we need a solution, something like, throttle the migration at the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/90f767adbb1901d007ee4dd8714f53402d893669/src/mongo/db/commands/tenant_migration_recipient_cmds.cpp#L74&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;command layer &lt;/a&gt; on the recipient side (i.e) Make recipeintSyncData cmd to wait if there are already 3 active concurrent migration in progress (for a default thread pool size 8) before asking POS to start a new migration&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="1602826">SERVER-54090</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1613403">SERVER-54328</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 18 Dec 2020 20:15:27 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 51 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-54090'>SERVER-54090</a></s>, <s><a href='https://jira.mongodb.org/browse/SERVER-54328'>SERVER-54328</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1791</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 51 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>andrew.shuvalov@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>lingzhi.deng@mongodb.com</customfieldvalue>
            <customfieldvalue>suganthi.mani@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hyjsv3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hy5gg7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4622">Repl 2021-02-08</customfieldvalue>
    <customfieldvalue id="4655">Repl 2021-02-22</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hyjf4f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>