<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:49:52 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-38744] w:majority collection drops may try to acquire database X lock after completing successfully</title>
                <link>https://jira.mongodb.org/browse/SERVER-38744</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;When a collection is dropped with w:majority, the server is supposed to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/f54e7ff52da1d6b20d30b4f9eb5d2f7f7bf7713a/src/mongo/db/repl/replication_coordinator_impl.cpp#L1542-L1555&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;wait internally for that drop and all earlier pending collection drops&lt;/a&gt; to physically complete i.e. to completely drop the collection from the catalog. This guarantees that no strong database locks will be taken as a result of asynchronous collection drops after a w:majority drop is acknowledged. In some cases, though, it is possible that a MODE_X database lock may be taken as a result of an asynchronous collection drop &lt;em&gt;after&lt;/em&gt; a w:majority collection drop was already acknowledged to a client. &lt;/p&gt;

&lt;p&gt;Consider the case of two collections in the same database, &lt;tt&gt;test.coll1&lt;/tt&gt; and &lt;tt&gt;test.coll2&lt;/tt&gt;. Let&apos;s say a client drops &lt;tt&gt;test.coll1&lt;/tt&gt; at w:1 and then drops &lt;tt&gt;test.coll2&lt;/tt&gt; at w:majority. By the time the drop of &lt;tt&gt;coll2&lt;/tt&gt; completes, it should be guaranteed that both &lt;tt&gt;test.coll1&lt;/tt&gt; and &lt;tt&gt;test.coll2&lt;/tt&gt; are physically dropped. It may still be possible for us to acquire a database lock after the &lt;tt&gt;coll2&lt;/tt&gt; drop completes, though. Whenever our commit point advances, we &lt;a href=&quot;https://github.com/mongodb/mongo/blob/f54e7ff52da1d6b20d30b4f9eb5d2f7f7bf7713a/src/mongo/db/repl/replication_coordinator_impl.cpp#L3290&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;notify oplog waiters&lt;/a&gt;, which in turn causes us to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/f54e7ff52da1d6b20d30b4f9eb5d2f7f7bf7713a/src/mongo/db/repl/replication_coordinator_external_state_impl.cpp#L874-L888&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;schedule a call&lt;/a&gt; to &lt;tt&gt;DropPendingCollectionReaper::dropCollectionsOlderThan&lt;/tt&gt;, if there exist drop pending collections and our new commit point is greater than or equal to the earliest drop pending optime. So, in the case of our two collections, we may advance our commit point to the optime of the &lt;tt&gt;test.coll1&lt;/tt&gt; drop, which would schedule such a call. We may then subsequently advance our commit point to the optime of the &lt;tt&gt;test.coll2&lt;/tt&gt; drop. We now have two outstanding calls scheduled to &lt;tt&gt;dropCollectionsOlderThan&lt;/tt&gt;. It is possible that the first scheduled call started to run and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/f54e7ff52da1d6b20d30b4f9eb5d2f7f7bf7713a/src/mongo/db/repl/drop_pending_collection_reaper.cpp#L138-L146&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;looked at the list of drop pending namespaces&lt;/a&gt;, thinking that it had to drop 1 collection i.e. &lt;tt&gt;test.coll1&lt;/tt&gt;. The second scheduled call, though, may end up running first, and drop both drop pending collections i.e. &lt;tt&gt;test.coll1&lt;/tt&gt; and &lt;tt&gt;test.coll2&lt;/tt&gt;. Because there are now no drop pending collections at this point, a majority write should be &lt;a href=&quot;https://github.com/mongodb/mongo/blob/f54e7ff52da1d6b20d30b4f9eb5d2f7f7bf7713a/src/mongo/db/repl/replication_coordinator_impl.cpp#L1542-L1555&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;allowed to complete&lt;/a&gt; and return to the client. The originally scheduled call to dropCollectionsOlderThan, though, may then complete, and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d0912b94ee141d46c14c00ba30d28037d5464af0/src/mongo/db/repl/drop_pending_collection_reaper.cpp#L156-L168&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;go through its list of collections&lt;/a&gt; (it only has one) and try to drop them. Since &lt;tt&gt;coll1&lt;/tt&gt; was already physically dropped, this will have no effect, but it will still try to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d0912b94ee141d46c14c00ba30d28037d5464af0/src/mongo/db/repl/storage_interface_impl.cpp#L458&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;acquire a MODE_X lock&lt;/a&gt; on the database, even after the w:majority collection drop completed and returned to the client.&lt;/p&gt;</description>
                <environment></environment>
        <key id="660167">SERVER-38744</key>
            <summary>w:majority collection drops may try to acquire database X lock after completing successfully</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="xiangyu.yao@mongodb.com">Xiangyu Yao</assignee>
                                    <reporter username="william.schultz@mongodb.com">William Schultz</reporter>
                        <labels>
                    </labels>
                <created>Fri, 21 Dec 2018 19:08:59 +0000</created>
                <updated>Wed, 13 Mar 2019 20:28:40 +0000</updated>
                            <resolved>Mon, 28 Jan 2019 22:33:17 +0000</resolved>
                                                                    <component>Replication</component>
                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>11</watches>
                                                                                                                <comments>
                            <comment id="2129996" author="xiangyu.yao" created="Mon, 28 Jan 2019 22:33:17 +0000"  >&lt;p&gt;Closing this ticket. We might have future QW tickets that could potentially fix the EMRC=false case.&lt;/p&gt;</comment>
                            <comment id="2129773" author="xiangyu.yao" created="Mon, 28 Jan 2019 20:18:48 +0000"  >&lt;p&gt;In 4.2, we will use the new two-phase drop so this issue can never happen.&lt;br/&gt;
In 4.0 and 4.2 with EMRC=false, this can still happen. But there is no easy way to fix it because of the intrinsic flaws in the old two-phase drop algorithm.&lt;/p&gt;</comment>
                            <comment id="2119641" author="geert.bosch" created="Thu, 17 Jan 2019 23:45:38 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;, while this is relevant on 4.0 as well, I do not think we should backport this change. As you indicate, the implementation on 4.0 would have to be significantly different, and therefore would require extra 4.0-only work. Significant changes to locking semantics in stable releases are risky. Any fix would affect all uses of MongoDB 4.0, while the locking behavior is really only an issue with applications that use interactive transactions, where the extra locking would degrade performance in the database where a collection is dropped.. &lt;/p&gt;</comment>
                            <comment id="2115525" author="milkie" created="Tue, 15 Jan 2019 14:45:46 +0000"  >&lt;p&gt;In the description of this ticket, it doesn&apos;t explain why the acquisition of the MODE_X lock is undesirable.  Does the undesirability extend to 4.0 and 3.6?&lt;/p&gt;</comment>
                            <comment id="2115509" author="judah.schvimer" created="Tue, 15 Jan 2019 14:39:00 +0000"  >&lt;p&gt;I think that this can happen on 4.0 as well (I don&apos;t think 3.6, but maybe). Do we want to backport it? I expect a backport might look significantly different from the master version.&lt;/p&gt;</comment>
                            <comment id="2114507" author="geert.bosch" created="Mon, 14 Jan 2019 18:44:25 +0000"  >&lt;p&gt;Indeed. It would make sense to have this be part of that project. &lt;/p&gt;</comment>
                            <comment id="2114506" author="tess.avitabile" created="Mon, 14 Jan 2019 18:42:35 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=geert.bosch&quot; class=&quot;user-hover&quot; rel=&quot;geert.bosch&quot;&gt;geert.bosch&lt;/a&gt;, will this issue go away once we do two-phase drops in WT? My understanding is that after we have two-phase drops in WT, we will never take locks for a drop after the drop command returns.&lt;/p&gt;</comment>
                            <comment id="2098784" author="william.schultz" created="Fri, 21 Dec 2018 19:15:20 +0000"  >&lt;p&gt;Note that this appeared as an issue for transactions related tests, since many of them rely on the fact that a w:majority drop guarantees no more strong locks will be taken in the middle of the test, thus interfering and potentially deadlocking with locks taken by transactions.&lt;/p&gt;</comment>
                            <comment id="2098781" author="william.schultz" created="Fri, 21 Dec 2018 19:14:01 +0000"  >&lt;p&gt;This is somewhat difficult to reproduce but it is possible to do by adding sleeps in the server.  By adding a sleep of around 100 milliseconds at &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d0912b94ee141d46c14c00ba30d28037d5464af0/src/mongo/db/repl/drop_pending_collection_reaper.cpp#L150&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this line&lt;/a&gt; and running the attached script,  &lt;span class=&quot;nobr&quot;&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/attachment/204565/204565_two_phase_drop_locks.js&quot; title=&quot;two_phase_drop_locks.js attached to SERVER-38744&quot;&gt;two_phase_drop_locks.js&lt;sup&gt;&lt;img class=&quot;rendericon&quot; src=&quot;https://jira.mongodb.org/images/icons/link_attachment_7.gif&quot; height=&quot;7&quot; width=&quot;7&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; , it is possible to see this issue appear. If there are two scheduled calls to &lt;tt&gt;dropCollectionsOlderThan&lt;/tt&gt;, where the first one tries to drop 1 collection and the second tries to drop 2 collections, the issue can be exposed by sleeping a while during the first call, but not the second. One way to trigger this is by sleeping randomly 0 or 100 milliseconds at the mentioned point in the code. After several runs, it becomes fairly likely to see a case where the first scheduled thread sleeps 100 milliseconds and the second scheduled thread sleeps 0 milliseconds, causing an X lock acquisition well after the w:majority drop already succeeded.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="523439">SERVER-34372</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="204565" name="two_phase_drop_locks.js" size="685" author="william.schultz@mongodb.com" created="Fri, 21 Dec 2018 19:10:39 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 14 Jan 2019 18:42:35 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 2 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1114</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>mikhail.shchatko@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 2 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>55.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>geert.bosch@mongodb.com</customfieldvalue>
            <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>
            <customfieldvalue>tess.avitabile@mongodb.com</customfieldvalue>
            <customfieldvalue>william.schultz@mongodb.com</customfieldvalue>
            <customfieldvalue>xiangyu.yao@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huhbgv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hu7clr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2761">Storage NYC 2019-02-11</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hugxq7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>