<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:43:59 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-36751] Prevent concurrent dropDatabase commands in the concurrency_simultaneous_replication suite</title>
                <link>https://jira.mongodb.org/browse/SERVER-36751</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;&lt;b&gt;Problem&lt;/b&gt;&lt;br/&gt;
 In the &lt;tt&gt;concurrency_simultaneous_replication&lt;/tt&gt; test suite, we run 10 operations in parallel on the same database, there&apos;s a small chance (e.g. 5% for some workloads) that an operation could be a &lt;tt&gt;dropDatabase&lt;/tt&gt;. For slower build variants, a single &lt;tt&gt;dropDatabase&lt;/tt&gt; command can take multiple minutes to finish if there is heavy activity from other workloads that are happening in parallel.&lt;/p&gt;

&lt;p&gt;Our tests will &lt;a href=&quot;https://github.com/mongodb/mongo/blob/17686781044525b9c3fbdf06ca326c8f4fb383ba/jstests/libs/override_methods/implicitly_retry_on_database_drop_pending.js#L148&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;retry an operation for up to 10 minutes&lt;/a&gt; if &lt;tt&gt;DatabaseDropPending&lt;/tt&gt; errors are encountered. After seeing the error, &lt;a href=&quot;https://github.com/mongodb/mongo/blob/17686781044525b9c3fbdf06ca326c8f4fb383ba/jstests/libs/override_methods/implicitly_retry_on_database_drop_pending.js#L25&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;A &lt;tt&gt;getLastError&lt;/tt&gt; command is used&lt;/a&gt; to wait for the &lt;tt&gt;dropDatabase&lt;/tt&gt; command to be committed.&lt;/p&gt;

&lt;p&gt;There is variability in the order that &lt;tt&gt;getLastError&lt;/tt&gt; returns from different workload clients, which may cause certain workload clients to always be stuck behind other clients that are doing more &lt;tt&gt;dropDatabase&lt;/tt&gt; commands. When this happens, the client will receive another &lt;tt&gt;DatabaseDropPending&lt;/tt&gt; error. But the client is &lt;a href=&quot;https://github.com/mongodb/mongo/blob/17686781044525b9c3fbdf06ca326c8f4fb383ba/jstests/libs/override_methods/implicitly_retry_on_database_drop_pending.js#L141&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;unable to distinguish&lt;/a&gt; whether the error is caused by the same dropDatabase command or a new one, causing the new wait to continue eat into the 10 minute timeout. There is a small probability that this cycle will happen for a handful of times in a row, which when combined with slow multi-minute &lt;tt&gt;dropDatabase&lt;/tt&gt; commands, will exceed the 10 minute timeout.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Solution&lt;/b&gt;&lt;br/&gt;
 The solution is to avoid retrying &lt;tt&gt;dropDatabase&lt;/tt&gt; commands when it returns a &lt;tt&gt;DatabaseDropPending&lt;/tt&gt; error. This will cause the workload to transition to a new state and continue to do so until the new state is no longer a &lt;tt&gt;dropDatabase&lt;/tt&gt; call. Then it will wait on the ongoing &lt;tt&gt;dropDatabase&lt;/tt&gt; call.&lt;/p&gt;

&lt;p&gt;When the database is finally dropped, it&apos;s guaranteed that none of the clients waiting on it would be another drop database, so they should all be able to proceed. There might be edge cases where one client is able to execute multiple commands and one of those commands is another &lt;tt&gt;dropDatabase&lt;/tt&gt;, but the likelihood of this happening 5 times in a row should be much smaller if not negligible.&lt;/p&gt;

&lt;p&gt;From a correctness perspective, this change will make some &lt;tt&gt;dropDatabase&lt;/tt&gt;&#160;implicitly into no-ops, which should not cause loss of test coverage, as databases can&apos;t be dropped in parallel in the first place. The tests that run parallel dropDatabases also all randomized tests and don&apos;t expect these operations to all succeed when there are parallel clients operating on the same database.&lt;/p&gt;

&lt;p&gt;&lt;del&gt;We should also write a dedicated regression test that does a high number of collection DDL operations while dropping and creating databases to simulate the timeout failures we&apos;ve seen, the changes from this ticket should prevent the test from failing.&lt;/del&gt;&lt;/p&gt;

&lt;p&gt;The &lt;del&gt;new test and the&lt;/del&gt; changes to not retry &lt;tt&gt;dropDatabase&lt;/tt&gt; should be limited to affect only the &lt;tt&gt;concurrency_simultaneous_replication&lt;/tt&gt; suite, as we have not seen this failure elsewhere so far.&lt;/p&gt;</description>
                <environment></environment>
        <key id="590318">SERVER-36751</key>
            <summary>Prevent concurrent dropDatabase commands in the concurrency_simultaneous_replication suite</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="max.hirschhorn@mongodb.com">Max Hirschhorn</assignee>
                                    <reporter username="robert.guo@mongodb.com">Robert Guo</reporter>
                        <labels>
                            <label>tig-bfday-eligible</label>
                            <label>tig-concurrency</label>
                    </labels>
                <created>Fri, 17 Aug 2018 21:26:43 +0000</created>
                <updated>Sun, 29 Oct 2023 22:28:48 +0000</updated>
                            <resolved>Wed, 26 Dec 2018 20:08:32 +0000</resolved>
                                                    <fixVersion>4.1.7</fixVersion>
                                    <component>Testing Infrastructure</component>
                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="2100451" author="xgen-internal-githook" created="Wed, 26 Dec 2018 20:08:13 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;username&apos;: &apos;visemet&apos;, &apos;email&apos;: &apos;max.hirschhorn@mongodb.com&apos;, &apos;name&apos;: &apos;Max Hirschhorn&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36751&quot; title=&quot;Prevent concurrent dropDatabase commands in the concurrency_simultaneous_replication suite&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-36751&quot;&gt;&lt;del&gt;SERVER-36751&lt;/del&gt;&lt;/a&gt; Skip retrying dropDatabase on DatabaseDropPending error.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/62c7e599ba211209eb93ae8f652d17fc8f6c251f&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/62c7e599ba211209eb93ae8f652d17fc8f6c251f&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2099544" author="max.hirschhorn@10gen.com" created="Mon, 24 Dec 2018 07:22:53 +0000"  >&lt;blockquote&gt;
&lt;p&gt;We should also write a dedicated regression test that does a high number of collection DDL operations while dropping and creating databases to simulate the timeout failures we&apos;ve seen, the changes from this ticket should prevent the test from failing.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;I don&apos;t think such a test case is going to be practical to run in Evergreen on a continuous basis.&lt;/p&gt;</comment>
                            <comment id="1989939" author="max.hirschhorn@10gen.com" created="Thu, 30 Aug 2018 14:55:00 +0000"  >&lt;p&gt;We can put the new behavior behind a &lt;tt&gt;TestData&lt;/tt&gt; option that&apos;s only enabled for the concurrency framework if there are other users of the &lt;tt&gt;implicitly_retry_on_database_drop_pending.js&lt;/tt&gt; override file outside of the concurrency framework.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="15640"><![CDATA[v4.0]]></customfieldvalue>
    <customfieldvalue key="15141"><![CDATA[v3.6]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 30 Aug 2018 14:55:00 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 7 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 7 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>
            <customfieldvalue>robert.guo@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu5tzr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr817j:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2694">STM 2018-12-31</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10555" key="com.atlassian.jira.plugin.system.customfieldtypes:float">
                        <customfieldname>Story Points</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu5g93:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>