<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:43:20 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-36514] Hold lock on oplog as soon as optime is reserved</title>
                <link>https://jira.mongodb.org/browse/SERVER-36514</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;There exists a potential 3-way deadlock currently.&lt;/p&gt;

&lt;p&gt;1. First, perform an insert into a replicated collection using &lt;a href=&quot;https://github.com/mongodb/mongo/blob/2c6bf9ead4270ffce87eda37b032848e89f86bc7/src/mongo/db/ops/write_ops_exec.cpp#L334-L357&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;insertDocuments()&lt;/a&gt;. An optime is generated, but not committed. If another write transaction commits after this with a later optime, a &quot;hole&quot; is created by the timestamped write that is not yet committed.&lt;/p&gt;

&lt;p&gt;2. Someone tries to read the oplog, for example a secondary querying against its sync source.  Oplog reads call &lt;a href=&quot;https://github.com/mongodb/mongo/blob/b65f5147a512cd57a323eb5853a503d8cf1bb557/src/mongo/db/exec/collection_scan.cpp#L120&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;waitForAllEarlierOplogWritesToBeVisible()&lt;/a&gt; to wait for all uncommitted operations to become committed and visible. &lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;This waits for the uncommitted insert in step 1 to be committed while holding an IS lock on the oplog (and thus on the &apos;local&apos; database).&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;3. A DDL operation is received on the &quot;local&quot; database, and enqueues a request for an X lock on the &apos;local&apos; database.&lt;/p&gt;


&lt;p&gt;Now the local database DDL operation in thread 3 blocks behind the IX lock held by thread 2.  Thread 2 can&apos;t complete until the insert in thread 1 completes.  And the insert in thread 1 winds up blocking when it goes to acquire an IX lock on the local database in order to write its oplog entry.  That lock acquisition is blocked by the pending X lock request by thread 3.&lt;/p&gt;

&lt;p&gt;This deadlock was made far more likely to hit as part of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-35365&quot; title=&quot;MapReduce temporary inc collections should be written to the local database&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-35365&quot;&gt;&lt;del&gt;SERVER-35365&lt;/del&gt;&lt;/a&gt;, which changed all mapReduce commands to perform DDL operations on the local database.&lt;/p&gt;

&lt;p&gt;One way to fix this would be if thread 1 took the lock on the oplog &lt;b&gt;before&lt;/b&gt; creating the oplog hole and held on to that lock until the oplog entry was written and the hole removed.&lt;/p&gt;</description>
                <environment></environment>
        <key id="585251">SERVER-36514</key>
            <summary>Hold lock on oplog as soon as optime is reserved</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="spencer@mongodb.com">Spencer Brody</assignee>
                                    <reporter username="spencer@mongodb.com">Spencer Brody</reporter>
                        <labels>
                    </labels>
                <created>Tue, 7 Aug 2018 22:23:59 +0000</created>
                <updated>Sat, 15 Sep 2018 14:48:55 +0000</updated>
                            <resolved>Fri, 24 Aug 2018 20:19:15 +0000</resolved>
                                                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>9</watches>
                                                                                                                <comments>
                            <comment id="1985447" author="spencer" created="Fri, 24 Aug 2018 20:19:15 +0000"  >&lt;p&gt;We fixed the deadlock by implementing &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36534&quot; title=&quot;Don&amp;#39;t acquire locks on oplog when writing oplog entries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-36534&quot;&gt;&lt;del&gt;SERVER-36534&lt;/del&gt;&lt;/a&gt; instead of this.&lt;/p&gt;</comment>
                            <comment id="1985117" author="greg.mckeon" created="Fri, 24 Aug 2018 17:05:44 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=spencer&quot; class=&quot;user-hover&quot; rel=&quot;spencer&quot;&gt;spencer&lt;/a&gt; is this still DwS?&lt;/p&gt;</comment>
                            <comment id="1970727" author="milkie" created="Wed, 8 Aug 2018 19:10:43 +0000"  >&lt;p&gt;Alternatively, we could go the other way and remove the oplog locking in logOp() at the Database and Collection level.&lt;/p&gt;</comment>
                            <comment id="1969789" author="spencer" created="Tue, 7 Aug 2018 22:27:11 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-35367&quot; title=&quot;Hold locks in fewer callers of waitForAllEarlierOplogWritesToBeVisible()&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-35367&quot;&gt;&lt;del&gt;SERVER-35367&lt;/del&gt;&lt;/a&gt; describes an alternate approach to fixing the same deadlock.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="553274">SERVER-35367</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="585626">SERVER-36534</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 8 Aug 2018 19:10:43 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 24 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>backlog-server-pm</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 24 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>greg.mckeon@mongodb.com</customfieldvalue>
            <customfieldvalue>spencer@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu4z2n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htvkgf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu4lbz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>