<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:39:37 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-35367] Hold locks in fewer callers of waitForAllEarlierOplogWritesToBeVisible()</title>
                <link>https://jira.mongodb.org/browse/SERVER-35367</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/b65f5147a512cd57a323eb5853a503d8cf1bb557/src/mongo/db/repl/replication_coordinator_external_state_impl.cpp#L449-L453&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;ReplicationCoordinatorExternalStateImpl::waitForAllEarlierOplogWritesToBeVisible() &lt;/a&gt; holds a collection lock on the oplog while doing a blocking wait. This can cause a hang described below:&lt;/p&gt;

&lt;p&gt;1. First, perform an insert into a replicated collection using &lt;a href=&quot;https://github.com/mongodb/mongo/blob/2c6bf9ead4270ffce87eda37b032848e89f86bc7/src/mongo/db/ops/write_ops_exec.cpp#L334-L357&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;insertDocuments()&lt;/a&gt;. An optime is generated, but not committed. If another write occurs after this at a later optime, a &quot;hole&quot; is created by the timestamped write is that is not yet committed.&lt;/p&gt;

&lt;p&gt;2. A reader using readConcern &quot;atClusterTime&quot; or &quot;afterClusterTime&quot; begins a read. This uses &lt;a href=&quot;https://github.com/mongodb/mongo/blob/b65f5147a512cd57a323eb5853a503d8cf1bb557/src/mongo/db/repl/replication_coordinator_external_state_impl.cpp#L449-L453&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;ReplicationCoordinatorExternalStateImpl::waitForAllEarlierOplogWritesToBeVisible()&lt;/a&gt; to wait for all uncommitted operations to become committed and visible.&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;This waits for the uncommitted insert in step 1 to be commited while holding a DBLock(&quot;local&quot;, MODE_IS)&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;3. A dropCollection command is received on the &quot;local&quot; database, and enqueues a DBLock(&quot;local&quot;, MODE_X).&lt;/p&gt;

&lt;p&gt;4. The first insert completes the insert in the storage engine and attempts to write the oplog entry at the generated optime. It attempts to acquire a DBLock(&quot;local&quot;, MODE_IX).&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;The previously enqueued dropCollection operation prevents the insert from acquiring the &quot;local&quot; database lock.&lt;/li&gt;
	&lt;li&gt;waitForAllEarlierOplogWritesToBeVisible() holds its collection lock while waiting the insert to become visible, which waits behind the dropCollection operation&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;This method should be redesigned so that a collection lock is not required to be held while waiting for the last oplog entry to become visible.&lt;/p&gt;</description>
                <environment></environment>
        <key id="553274">SERVER-35367</key>
            <summary>Hold locks in fewer callers of waitForAllEarlierOplogWritesToBeVisible()</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="spencer@mongodb.com">Spencer Brody</assignee>
                                    <reporter username="louis.williams@mongodb.com">Louis Williams</reporter>
                        <labels>
                            <label>SWNA</label>
                            <label>nyc</label>
                    </labels>
                <created>Fri, 1 Jun 2018 21:03:21 +0000</created>
                <updated>Sun, 29 Oct 2023 22:31:09 +0000</updated>
                            <resolved>Wed, 15 Aug 2018 21:25:03 +0000</resolved>
                                    <version>4.0.0-rc1</version>
                    <version>4.1.1</version>
                                    <fixVersion>4.0.2</fixVersion>
                    <fixVersion>4.1.2</fixVersion>
                                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>14</watches>
                                                                                                                <comments>
                            <comment id="1978145" author="xgen-internal-githook" created="Thu, 16 Aug 2018 21:57:29 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;username&apos;: &apos;stbrody&apos;, &apos;email&apos;: &apos;spencer@mongodb.com&apos;, &apos;name&apos;: &apos;Spencer T Brody&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-35367&quot; title=&quot;Hold locks in fewer callers of waitForAllEarlierOplogWritesToBeVisible()&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-35367&quot;&gt;&lt;del&gt;SERVER-35367&lt;/del&gt;&lt;/a&gt; Do not hold locks while calling waitForAllEarlierOplogWritesToBeVisible()&lt;/p&gt;

&lt;p&gt;(cherry picked from commit a1b225bcf0e9791b14649df385b3f3f9710a98ab)&lt;br/&gt;
Branch: v4.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/fff8892f9d2420d9ea4d248c4a9f708d03a883f9&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/fff8892f9d2420d9ea4d248c4a9f708d03a883f9&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1976892" author="spencer" created="Wed, 15 Aug 2018 21:24:48 +0000"  >&lt;p&gt;Updating the title of this ticket to reflect the work that was actually done.  While some callers of waitForAllEarlierOplogWritesToBeVisible no longer hold locks on the oplog, there are still a few that do.  We&apos;d like to get rid of all of such places, but that is a larger change than we are ready to make at the moment.  In the meantime we hope to address the deadlock described in this ticket via &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36534&quot; title=&quot;Don&amp;#39;t acquire locks on oplog when writing oplog entries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-36534&quot;&gt;&lt;del&gt;SERVER-36534&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="1969790" author="spencer" created="Tue, 7 Aug 2018 22:27:31 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36514&quot; title=&quot;Hold lock on oplog as soon as optime is reserved&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-36514&quot;&gt;&lt;del&gt;SERVER-36514&lt;/del&gt;&lt;/a&gt; describes an alternate approach to fixing the same deadlock.&lt;/p&gt;</comment>
                            <comment id="1966528" author="spencer" created="Fri, 3 Aug 2018 17:54:53 +0000"  >&lt;p&gt;My previous fix is incomplete, it doesn&apos;t address oplog queries used for replication that call into waitForEarlierOplogWritesToBeVisible while holding locks on the oplog via the normal query execution machinery.&lt;/p&gt;</comment>
                            <comment id="1961690" author="xgen-internal-githook" created="Tue, 31 Jul 2018 15:55:41 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Spencer T Brody&apos;, &apos;email&apos;: &apos;spencer@mongodb.com&apos;, &apos;username&apos;: &apos;stbrody&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-35367&quot; title=&quot;Hold locks in fewer callers of waitForAllEarlierOplogWritesToBeVisible()&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-35367&quot;&gt;&lt;del&gt;SERVER-35367&lt;/del&gt;&lt;/a&gt; Do not hold locks while calling waitForAllEarlierOplogWritesToBeVisible()&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/a1b225bcf0e9791b14649df385b3f3f9710a98ab&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/a1b225bcf0e9791b14649df385b3f3f9710a98ab&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1938119" author="milkie" created="Tue, 3 Jul 2018 18:14:14 +0000"  >&lt;p&gt;I checked and all the code that participates in this bug is present in 3.6 so I assume it does exist there as well.&lt;/p&gt;</comment>
                            <comment id="1938113" author="tess.avitabile" created="Tue, 3 Jul 2018 18:10:29 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt;, do you know if this bug exists on 3.6?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="585175">SERVER-36508</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10520">
                    <name>Problem/Incident</name>
                                            <outwardlinks description="causes">
                                        <issuelink>
            <issuekey id="601274">SERVER-37048</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="731336">SERVER-40498</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="553260">SERVER-35365</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="585251">SERVER-36514</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="585626">SERVER-36534</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="15640"><![CDATA[v4.0]]></customfieldvalue>
    <customfieldvalue key="15141"><![CDATA[v3.6]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[500A000000cEmVWIA0, 5002K00000pDjJSQA0]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 3 Jul 2018 18:10:29 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 25 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-36508'>SERVER-36508</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 25 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>36.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>
            <customfieldvalue>spencer@mongodb.com</customfieldvalue>
            <customfieldvalue>tess.avitabile@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htzmj3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr8qsv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2384">Repl 2018-07-30</customfieldvalue>
    <customfieldvalue id="2447">Repl 2018-08-13</customfieldvalue>
    <customfieldvalue id="2448">Repl 2018-08-27</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htz8sf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>