<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:43:06 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-36439] Fix the concurrency control of the LogicalTimeValidator</title>
                <link>https://jira.mongodb.org/browse/SERVER-36439</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;The concurrency control of the LogicalTimeValidator with respect to the KeyManager is not fully correct. Specifically, the _mutexKeyManager seems to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/1d2038b2b84172abbe06ead3a2fa80f833998651/src/mongo/db/logical_time_validator.cpp#L111&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;serve more the purpose&lt;/a&gt; of a barrier than a mutex. Most likely, because the KeyManager is self-synchronizing we don&apos;t need the &lt;tt&gt;_mutexKeyManager&lt;/tt&gt;.&lt;/p&gt;

&lt;p&gt;As implemented currently, a deadlock can occur between LogicalTimeValidator::shutDown() and LogicalTimeValidator::_getKeyManagerCopy() if a periodic refresh occurs during shutdown.&lt;/p&gt;</description>
                <environment></environment>
        <key id="583201">SERVER-36439</key>
            <summary>Fix the concurrency control of the LogicalTimeValidator</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="anton.oyung@mongodb.com">Anton Oyung</assignee>
                                    <reporter username="janna.golden@mongodb.com">Janna Golden</reporter>
                        <labels>
                    </labels>
                <created>Fri, 3 Aug 2018 17:44:09 +0000</created>
                <updated>Sun, 29 Oct 2023 22:29:18 +0000</updated>
                            <resolved>Tue, 19 Nov 2019 16:37:32 +0000</resolved>
                                                    <fixVersion>4.3.2</fixVersion>
                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="2590223" author="misha.tyulenev" created="Fri, 6 Dec 2019 18:51:43 +0000"  >&lt;p&gt;The concurrency issue just showed up in BF-15613. There the deadlock occurred when there were a message that mongos wanted to send while it was in shutdown. &lt;br/&gt;
The trySignLogicalTime needs _getKeyManagerCopy that &lt;a href=&quot;https://github.com/mongodb/mongo/blob/v3.6/src/mongo/db/logical_time_validator.cpp#L228&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;takes a lock &lt;/a&gt;&lt;br/&gt;
But the lock is already taken as &lt;a href=&quot;https://github.com/mongodb/mongo/blob/v3.6/src/mongo/db/logical_time_validator.cpp#L178&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;its in shutdown&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="2549175" author="xgen-internal-githook" created="Tue, 19 Nov 2019 15:51:52 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;username&apos;: &apos;AntonOyung&apos;, &apos;email&apos;: &apos;anton.oyung@mongodb.com&apos;, &apos;name&apos;: &apos;Anton Oyung&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36439&quot; title=&quot;Fix the concurrency control of the LogicalTimeValidator&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-36439&quot;&gt;&lt;del&gt;SERVER-36439&lt;/del&gt;&lt;/a&gt; Fix the concurrency control of the LogicalTimeValidator&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/5b73c31741584597bae0455a50364513c05dc1d1&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/5b73c31741584597bae0455a50364513c05dc1d1&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2540380" author="kaloian.manassiev" created="Thu, 14 Nov 2019 09:54:04 +0000"  >&lt;p&gt;In that case yes, I think it is safe to throw out this mutex. Thanks, Anton.&lt;/p&gt;</comment>
                            <comment id="2538789" author="anton.oyung" created="Wed, 13 Nov 2019 14:54:03 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev&quot;&gt;kaloian.manassiev&lt;/a&gt; yes that&apos;s correct. The idea of adding the extra mutex on the KeyCollectionManager is in case future functionality is added to it that could possibly cause concurrency issues in the future. But maybe it&apos;s not necessary to implement that now.&#160;&lt;/p&gt;</comment>
                            <comment id="2538653" author="kaloian.manassiev" created="Wed, 13 Nov 2019 13:23:36 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=anton.oyung&quot; class=&quot;user-hover&quot; rel=&quot;anton.oyung&quot;&gt;anton.oyung&lt;/a&gt;, what you are saying is that &lt;tt&gt;_mutexKeyManager&lt;/tt&gt; protects the setting of the &lt;tt&gt;_keyManager&lt;/tt&gt; field, but because it is only set in the constructor once, we can get rid of the mutex altogether. Do I understand it correctly?&lt;/p&gt;

&lt;p&gt;Because I see also that the &lt;tt&gt;stopMonitoring&lt;/tt&gt;, &lt;tt&gt;clearCache&lt;/tt&gt; and &lt;tt&gt;resetCache&lt;/tt&gt; methods are being called under that mutex as well. However, given that the KeyManager is supposed to be self-synchronising, I think we don&apos;t need a separate outside mutex to guard it as well and because of this we should be able to throw out &lt;tt&gt;_mutexKeyManager&lt;/tt&gt; indeed.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=misha.tyulenev&quot; class=&quot;user-hover&quot; rel=&quot;misha.tyulenev&quot;&gt;misha.tyulenev&lt;/a&gt;, does this ^ sound reasonable?&lt;/p&gt;</comment>
                            <comment id="2538097" author="anton.oyung" created="Tue, 12 Nov 2019 23:38:00 +0000"  >&lt;p&gt;It looks like at one point we used to call `_keyManager.reset()` here &lt;a href=&quot;https://github.com/mongodb/mongo/commit/eeee1e2b64f70e8487f017ba579f3ca861c81e4f#diff-78b4e45965b90ad3d214c8cc3159d89aR213&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/eeee1e2b64f70e8487f017ba579f3ca861c81e4f#diff-78b4e45965b90ad3d214c8cc3159d89aR213&lt;/a&gt; which required us to have `_mutexKeyManager`. Since then it&#8217;s been deleted so it seems like this getter with the mutex isn&#8217;t necessary and we can instead call _keyManager directly in all of the places the getter was used.&lt;/p&gt;

&lt;p&gt;Additionally from my investigation the `KeyCollectionManager` seems thread safe since both the refresher and cache are thread safe themselves. But it might be a good idea to add a mutex and acquire it whenever the `KeyCollectionManager` does something.&#160;&lt;/p&gt;

&lt;p&gt;Does this approach sound alright or is there something else I should consider?&lt;/p&gt;</comment>
                            <comment id="2240628" author="kaloian.manassiev" created="Thu, 9 May 2019 21:10:24 +0000"  >&lt;p&gt;The concurrency control of the LogicalTimeValidator with respect to the KeyManager looks pretty janky and is not fully correct. Specifically, the &lt;tt&gt;_mutexKeyManager&lt;/tt&gt; seems to serve more the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/1d2038b2b84172abbe06ead3a2fa80f833998651/src/mongo/db/logical_time_validator.cpp#L111&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;purpose of a barrier&lt;/a&gt; than a mutex. Because of this, I think the deadlock itself is the lesser problem, because it can only happen on shutdown.&lt;/p&gt;

&lt;p&gt;I am re-purposing this ticket to mean &quot;Fix the concurrency control of LogicalTimeValidator&quot;.&lt;/p&gt;</comment>
                            <comment id="2235562" author="misha.tyulenev" created="Mon, 6 May 2019 16:35:13 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev&quot;&gt;kaloian.manassiev&lt;/a&gt; the blocking calls are &lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/db/logical_time_validator.cpp#L201&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/master/src/mongo/db/logical_time_validator.cpp#L201&lt;/a&gt; and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/db/logical_time_validator.cpp#L211&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/master/src/mongo/db/logical_time_validator.cpp#L211&lt;/a&gt; . _getKeyManagerCopy will serialize on the _mutexKeyManager&lt;/p&gt;</comment>
                            <comment id="2235470" author="kaloian.manassiev" created="Mon, 6 May 2019 16:01:56 +0000"  >&lt;p&gt;The description on the ticket doesn&apos;t look right. The &lt;a href=&quot;https://github.com/mongodb/mongo/blob/da1c5de6be5ef90b127dfbdfe5473354bcf7cb7e/src/mongo/db/logical_time_validator.cpp#L229&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;LogicalTimeValidator::_getKeyManagerCopy()&lt;/tt&gt;&lt;/a&gt; call only takes a single mutex and doesn&apos;t do any blocking operation, so it cannot possibly be part of a deadlock cycle.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=misha.tyulenev&quot; class=&quot;user-hover&quot; rel=&quot;misha.tyulenev&quot;&gt;misha.tyulenev&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=janna.golden&quot; class=&quot;user-hover&quot; rel=&quot;janna.golden&quot;&gt;janna.golden&lt;/a&gt;: I think we should just throw out the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/da1c5de6be5ef90b127dfbdfe5473354bcf7cb7e/src/mongo/db/logical_time_validator.h#L127&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;_mutexKeyManager&lt;/tt&gt;&lt;/a&gt; since it doesn&apos;t do anything useful.&lt;/p&gt;

&lt;p&gt;In either case though, this is not an RC blocker.&lt;/p&gt;</comment>
                            <comment id="2066582" author="misha.tyulenev" created="Mon, 19 Nov 2018 17:15:26 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=greg.mckeon&quot; class=&quot;user-hover&quot; rel=&quot;greg.mckeon&quot;&gt;greg.mckeon&lt;/a&gt; I believe this will happen in all versions starting 3.6&lt;/p&gt;</comment>
                            <comment id="2066305" author="kaloian.manassiev" created="Mon, 19 Nov 2018 15:27:20 +0000"  >&lt;p&gt;This is probably more of a question for &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jack.mulrow&quot; class=&quot;user-hover&quot; rel=&quot;jack.mulrow&quot;&gt;jack.mulrow&lt;/a&gt; or &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=misha.tyulenev&quot; class=&quot;user-hover&quot; rel=&quot;misha.tyulenev&quot;&gt;misha.tyulenev&lt;/a&gt;. I have not looked at any of the call stacks and have no idea if any of them changed after 3.6.&lt;/p&gt;</comment>
                            <comment id="2066286" author="greg.mckeon" created="Mon, 19 Nov 2018 15:21:32 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=janna.golden&quot; class=&quot;user-hover&quot; rel=&quot;janna.golden&quot;&gt;janna.golden&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev&quot;&gt;kaloian.manassiev&lt;/a&gt; can this only happen in 3.6? Should it be 3.6 required?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>12.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="15640"><![CDATA[v4.0]]></customfieldvalue>
    <customfieldvalue key="15141"><![CDATA[v3.6]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 19 Nov 2018 15:21:32 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 9 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1489</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>false</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 9 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>58.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>anton.oyung@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>greg.mckeon@mongodb.com</customfieldvalue>
            <customfieldvalue>janna.golden@mongodb.com</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
            <customfieldvalue>misha.tyulenev@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu4mi7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hvln4n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2726">Sharding 2019-01-28</customfieldvalue>
    <customfieldvalue id="2786">Sharding 2019-02-11</customfieldvalue>
    <customfieldvalue id="3307">Sharding 2019-11-04</customfieldvalue>
    <customfieldvalue id="3308">Sharding 2019-11-18</customfieldvalue>
    <customfieldvalue id="3472">Sharding 2019-12-02</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu48rj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>