<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:52:30 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-39591] RecoveryUnit::waitUntilDurable() should invariant against callers holding locks</title>
                <link>https://jira.mongodb.org/browse/SERVER-39591</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;MMAP&apos;s implementation had an invariant, but the other storage engine implementations appear not to have any.&lt;/p&gt;

&lt;p&gt;Locks should not be held while waitUntilDurable is called for two reasons: 1) it can take a long time to make data durable, so a lock should not be held that long for I/O; 2) we could run out of tickets and the lock that the caller has could stop the rest of the system from freeing up enough for he durability task to get a ticket, thus a ticketing deadlock.&lt;/p&gt;

&lt;p&gt;This requires fixing the failpoint in index builds that calls waitUntilDurable() with locks, curtesy of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-39585&quot; title=&quot;Explore whether index build failpoint &amp;#39;crashAfterStartingIndexBuild&amp;#39; requires dropping locks to wait for durability&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-39585&quot;&gt;&lt;del&gt;SERVER-39585&lt;/del&gt;&lt;/a&gt; to unblock other work&lt;/p&gt;</description>
                <environment></environment>
        <key id="697915">SERVER-39591</key>
            <summary>RecoveryUnit::waitUntilDurable() should invariant against callers holding locks</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="dianna.hohensee@mongodb.com">Dianna Hohensee</assignee>
                                    <reporter username="dianna.hohensee@mongodb.com">Dianna Hohensee</reporter>
                        <labels>
                    </labels>
                <created>Thu, 14 Feb 2019 22:20:36 +0000</created>
                <updated>Sun, 29 Oct 2023 22:23:59 +0000</updated>
                            <resolved>Mon, 18 Nov 2019 17:53:29 +0000</resolved>
                                                    <fixVersion>4.3.1</fixVersion>
                                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="2547199" author="xgen-internal-githook" created="Mon, 18 Nov 2019 17:24:51 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;username&apos;: &apos;DiannaHohensee&apos;, &apos;email&apos;: &apos;dianna.hohensee@mongodb.com&apos;, &apos;name&apos;: &apos;Dianna Hohensee&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-39591&quot; title=&quot;RecoveryUnit::waitUntilDurable() should invariant against callers holding locks&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-39591&quot;&gt;&lt;del&gt;SERVER-39591&lt;/del&gt;&lt;/a&gt; RecoveryUnit::waitUntilDurable() invariants against callers holding locks, except for repair&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/e229612493fe002183a948da98982c08ce5ce24a&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/e229612493fe002183a948da98982c08ce5ce24a&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2474008" author="evgeni.dobranov" created="Wed, 9 Oct 2019 15:42:20 +0000"  >&lt;p&gt;We&apos;re closing out&#160;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-41463&quot; title=&quot;repairDatabasesAndCheckVersion should not hold a lock while calling on StorageRepairObserver::onRepairDone&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-41463&quot;&gt;&lt;del&gt;SERVER-41463&lt;/del&gt;&lt;/a&gt; (one of the dependent tickets that has to do with repair) without a fix. Instead of changing locking/unlocking or changing where waitUntilDurable is called there, let&apos;s upstream the fix to this ticket by adding an invariant as such:&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;&lt;tt&gt;invariant(!opCtx-&amp;gt;lockState()-&amp;gt;isLocked() &lt;b&gt;|| storageGlobalParams.repair&lt;/b&gt;)&lt;/tt&gt;&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;The full reasoning can be found &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-41463?focusedCommentId=2472418&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-2472418&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;in this comment&lt;/a&gt; under the original ticket.&lt;/p&gt;</comment>
                            <comment id="2253158" author="dianna.hohensee" created="Mon, 20 May 2019 18:53:52 +0000"  >&lt;p&gt;So we cannot just delete the waitUntilDurable() calls from setInitialSyncFlag and clearInitialSyncFlag. If we end initial sync calling clearInitialSyncFlag, and do not wait for durability, then the server can restart and fall back into initial syncing. This would be bad because initial sync is not resumable: it will start over completely.&lt;/p&gt;

&lt;p&gt;We would have to do some work to explore/try elevating the waitUntilDurable() calls to a higher level without locks.&lt;/p&gt;

&lt;p&gt;The waitUntilDurable() calls under a lock do no harm where they&apos;re currently called, I&apos;d imagine, because only initial sync is running on the server, so nothing else is blocked. But it does block us from invariant&apos;ing against future misuse of the waitUntilDurable() function.&lt;/p&gt;</comment>
                            <comment id="2250912" author="milkie" created="Fri, 17 May 2019 18:52:57 +0000"  >&lt;p&gt;I wonder if either of those locations needs that call.  Perhaps we can simply remove the waitUntilDurable() calls there.&lt;/p&gt;</comment>
                            <comment id="2250842" author="dianna.hohensee" created="Fri, 17 May 2019 18:16:17 +0000"  >&lt;p&gt;ReplicationConsistencyMarkersImpl::setInitialSyncFlag is called during initial sync, &lt;a href=&quot;https://github.com/mongodb/mongo/blob/6cebd61ff8d36e5b61e491419baa4808a98324a6/src/mongo/db/repl/replication_consistency_markers_impl.cpp#L139-L140&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;immediately after doing a write to a document&lt;/a&gt;. This seems hard to eradicate.&lt;/p&gt;</comment>
                            <comment id="2250825" author="dianna.hohensee" created="Fri, 17 May 2019 18:10:33 +0000"  >&lt;p&gt;repairDatabasesAndCheckVersion holds a Global X lock throughout. StorageRepairObserver::_invalidateReplConfigIfNeeded does a &lt;a href=&quot;https://github.com/mongodb/mongo/blob/6cebd61ff8d36e5b61e491419baa4808a98324a6/src/mongo/db/storage/storage_repair_observer.cpp#L149-L151&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;document write right before calling waitUntilDurable()&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="2250208" author="milkie" created="Fri, 17 May 2019 13:33:32 +0000"  >&lt;p&gt;repairDatabasesAndCheckVersion doesn&apos;t need to hold any locks while it calls waitUntilDurable.&lt;/p&gt;
</comment>
                            <comment id="2165192" author="xgen-internal-githook" created="Wed, 27 Feb 2019 20:12:30 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Dianna Hohensee&apos;, &apos;username&apos;: &apos;DiannaHohensee&apos;, &apos;email&apos;: &apos;dianna.hohensee@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-39591&quot; title=&quot;RecoveryUnit::waitUntilDurable() should invariant against callers holding locks&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-39591&quot;&gt;&lt;del&gt;SERVER-39591&lt;/del&gt;&lt;/a&gt; replace the one remaining use of the &apos;crashAfterStartingIndexBuild&apos; failpoint and delete the failpoint itself&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/24c08a88060a8f86e3c0af2022c853e845996eee&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/24c08a88060a8f86e3c0af2022c853e845996eee&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="787594">SERVER-41463</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="787607">SERVER-41464</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="795313">SERVER-41613</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 27 Feb 2019 20:12:30 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 12 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-41463'>SERVER-41463</a></s>, <s><a href='https://jira.mongodb.org/browse/SERVER-41464'>SERVER-41464</a></s>, <s><a href='https://jira.mongodb.org/browse/SERVER-41613'>SERVER-41613</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 12 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>dianna.hohensee@mongodb.com</customfieldvalue>
            <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>evgeni.dobranov@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hunm5b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr66yn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2909">Storage NYC 2019-05-20</customfieldvalue>
    <customfieldvalue id="3345">Execution Team 2019-11-18</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hun8en:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>