<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:41:46 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-57407] Avoid taking ReplicationCoordinator mutex when deciding to gossip cluster times with external clients</title>
                <link>https://jira.mongodb.org/browse/SERVER-57407</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;To decide whether to gossip cluster times to and from external clients, mongod will &lt;a href=&quot;https://github.com/mongodb/mongo/blob/898d47a64a9b98e018308bbcc2329b65702d9b3f/src/mongo/db/vector_clock_mongod.cpp#L72&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;check if it is in a readable state&lt;/a&gt;&#160;(as of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-47568&quot; title=&quot;No keys found for HMAC in RECOVERING state&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-47568&quot;&gt;&lt;del&gt;SERVER-47568&lt;/del&gt;&lt;/a&gt;). Validating incoming cluster times requires having cached signing keys from &lt;tt&gt;admin.system.keys&lt;/tt&gt; so this is meant to avoid persistent validation errors when a node enters an unreadable state.&lt;/p&gt;

&lt;p&gt;Currently, this check happens twice for each received external request and uses the ReplicationCoordinator method &lt;tt&gt;getMemberState()&lt;/tt&gt;. This method takes the ReplicationCoordinator mutex, which is often contested and can have a performance impact on certain workloads. Instead, a method that does not take this mutex should be used, like &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c63c85e2a0535ead596dd06fac71aa97976a3b76/src/mongo/db/repl/replication_coordinator.h#L209&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;ReplicationCoordinator::isInPrimaryOrSecondaryState()&lt;/tt&gt;&lt;/a&gt;.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1767885">SERVER-57407</key>
            <summary>Avoid taking ReplicationCoordinator mutex when deciding to gossip cluster times with external clients</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="randolph@mongodb.com">Randolph Tan</assignee>
                                    <reporter username="jack.mulrow@mongodb.com">Jack Mulrow</reporter>
                        <labels>
                            <label>sharding-wfbf-day</label>
                    </labels>
                <created>Thu, 3 Jun 2021 19:26:51 +0000</created>
                <updated>Sun, 29 Oct 2023 21:52:42 +0000</updated>
                            <resolved>Fri, 17 Sep 2021 16:54:32 +0000</resolved>
                                                    <fixVersion>5.1.0-rc0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>12</watches>
                                                                                                                <comments>
                            <comment id="4553860" author="renctan" created="Tue, 17 May 2022 17:24:38 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=qwan%40stripe.com&quot; class=&quot;user-hover&quot; rel=&quot;qwan@stripe.com&quot;&gt;qwan@stripe.com&lt;/a&gt; We didn&apos;t see a significant difference in the perf numbers in one of our workloads so we ended up not backporting it.&lt;/p&gt;</comment>
                            <comment id="4546382" author="JIRAUSER1262685" created="Thu, 12 May 2022 20:53:17 +0000"  >&lt;p&gt;This patch is said to be back-ported to v4.x, did it ever happen?&#160;&lt;/p&gt;</comment>
                            <comment id="4068842" author="xgen-internal-githook" created="Fri, 17 Sep 2021 15:53:22 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Randolph Tan&apos;, &apos;email&apos;: &apos;randolph@10gen.com&apos;, &apos;username&apos;: &apos;renctan&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-57407&quot; title=&quot;Avoid taking ReplicationCoordinator mutex when deciding to gossip cluster times with external clients&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-57407&quot;&gt;&lt;del&gt;SERVER-57407&lt;/del&gt;&lt;/a&gt; Avoid taking ReplicationCoordinator mutex when deciding &#8230;&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/6adde2f790fecd9f9401c11462ba14639d02cd60&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/6adde2f790fecd9f9401c11462ba14639d02cd60&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4039167" author="bruce.lucas@10gen.com" created="Fri, 3 Sep 2021 15:23:09 +0000"  >&lt;p&gt;I opened &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-59776&quot; title=&quot;50% regression in single multi-update&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-59776&quot;&gt;&lt;del&gt;SERVER-59776&lt;/del&gt;&lt;/a&gt; regarding the more general issue of replication coordinator mutex contention because it looked to me like this ticket is aimed at a more specific issue.&lt;/p&gt;</comment>
                            <comment id="3860120" author="JIRAUSER1256988" created="Fri, 4 Jun 2021 16:07:27 +0000"  >&lt;p&gt;I would like to stress that if there is a Mutex that is already known to experience substantial lock contention then avoiding to use this mutex in simple &quot;get()&quot; methods on critical path is kicking the can down the road. Besides doing this, an effort to actually reduce the contention must be made. Remember the maxima - &quot;lock contention is always a bug&quot;.&lt;/p&gt;

&lt;p&gt;Just after giving few minutes look at the Mutex I can immediately suggest to explore the possibility that the `_heartbeatHandles` collection &lt;a href=&quot;https://github.com/mongodb/mongo/blob/1e34385e2649de1d3844942b512ffcf7e89d0bc3/src/mongo/db/repl/replication_coordinator_impl_heartbeat.cpp#L880-L931&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;used in 4 methods only that tracks outstanding heartbeats&lt;/a&gt; is pretty autonomous and can have its own mutex. That will in its order allow to call the `_replExecutor-&amp;gt;scheduleRemoteCommand()` without holding any lock, which will allow nested locking because executor is already internally synchronized.&lt;/p&gt;

&lt;p&gt;So the suggestion is:&lt;br/&gt;
1. Open separate tickets&lt;br/&gt;
2. Assign the ReplicationCoordinatorImpl::_mutex an explicit high level, like 5&lt;br/&gt;
3. Add level 1 mutex for the _heartbeatHandles&lt;br/&gt;
4. Find the next low hanging fruit and repeat the cycle&lt;/p&gt;

</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10520">
                    <name>Problem/Incident</name>
                                            <outwardlinks description="causes">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="1867214">SERVER-59776</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="21777"><![CDATA[v5.0]]></customfieldvalue>
    <customfieldvalue key="18953"><![CDATA[v4.4]]></customfieldvalue>
    <customfieldvalue key="16775"><![CDATA[v4.2]]></customfieldvalue>
    <customfieldvalue key="15640"><![CDATA[v4.0]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[5002K00000xDlvMQAS]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 4 Jun 2021 16:07:27 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 38 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 38 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>135.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>andrew.shuvalov@mongodb.com</customfieldvalue>
            <customfieldvalue>bruce.lucas@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>jack.mulrow@mongodb.com</customfieldvalue>
            <customfieldvalue>qwan@stripe.com</customfieldvalue>
            <customfieldvalue>randolph@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzjkxz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hz46pj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4521">Sharding 2021-07-12</customfieldvalue>
    <customfieldvalue id="4522">Sharding 2021-07-26</customfieldvalue>
    <customfieldvalue id="5218">Sharding 2021-08-09</customfieldvalue>
    <customfieldvalue id="5219">Sharding 2021-08-23</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzj773:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>