<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:40:27 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-79245] Unclean shutdown while dropping collection and indexes to resync can make the catalog inconsistent</title>
                <link>https://jira.mongodb.org/browse/SERVER-79245</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Initial sync will drop all tables in all replicated databases without a timestamp before resyncing. This means that the drop pending ident reaper will immediately drop the table in WT the next time it runs. The table drops in WT are non-transactional and cannot be rolled back. This leads to immediately dropping the table in WT even if the catalog changes are not stable/checkpointed. As a result, during startup recovery, the table no longer exists in WT but continues to exist in the catalog. The server tries to query the index table metadata from WT but WT likely returns ENOENT and we crash.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2399111">SERVER-79245</key>
            <summary>Unclean shutdown while dropping collection and indexes to resync can make the catalog inconsistent</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="jordi.olivares-provencio@mongodb.com">Jordi Olivares Provencio</assignee>
                                    <reporter username="gregory.wlodarek@mongodb.com">Gregory Wlodarek</reporter>
                        <labels>
                    </labels>
                <created>Mon, 24 Jul 2023 13:42:11 +0000</created>
                <updated>Wed, 4 Oct 2023 21:12:43 +0000</updated>
                            <resolved>Thu, 28 Sep 2023 14:56:29 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>10</watches>
                                                                                                                <comments>
                            <comment id="5737742" author="JIRAUSER1264163" created="Thu, 28 Sep 2023 14:56:29 +0000"  >&lt;p&gt;The problem will be fixed in a more generic way by &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-80974&quot; title=&quot;Unclean shutdown while dropping local.* collection and indexes can make the catalog inconsistent&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-80974&quot;&gt;&lt;del&gt;SERVER-80974&lt;/del&gt;&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5724620" author="JIRAUSER1264163" created="Fri, 22 Sep 2023 16:08:30 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=suganthi.mani%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;suganthi.mani@mongodb.com&quot;&gt;suganthi.mani@mongodb.com&lt;/a&gt; I wals also thinking the same as I was working on &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-80974&quot; title=&quot;Unclean shutdown while dropping local.* collection and indexes can make the catalog inconsistent&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-80974&quot;&gt;&lt;del&gt;SERVER-80974&lt;/del&gt;&lt;/a&gt;. The general solution proposed there is to defer the drop until a checkpoint has occurred for the data.&lt;/p&gt;</comment>
                            <comment id="5724575" author="suganthi.mani" created="Fri, 22 Sep 2023 15:59:39 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jordi.olivares-provencio%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;jordi.olivares-provencio@mongodb.com&quot;&gt;jordi.olivares-provencio@mongodb.com&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=gregory.wlodarek%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;gregory.wlodarek@mongodb.com&quot;&gt;gregory.wlodarek@mongodb.com&lt;/a&gt; I filed &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-81355&quot; title=&quot;Unclean shutdown after untimestamped collection drop can make catalog inconsistent.&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-81355&quot;&gt;&lt;del&gt;SERVER-81355&lt;/del&gt;&lt;/a&gt; because we are encountering a similar issue with shard merge. My understanding is that there are many places in our codebase where we perform untimestamped drops of internal collections (See &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-75740&quot; title=&quot;system.profile collection is created using untimestamped writes but dropped using timestamped writes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-75740&quot;&gt;&lt;del&gt;SERVER-75740&lt;/del&gt;&lt;/a&gt;), and all of these places could have the same issue. Looking at PR 15460, it seems that the general idea is that the places performing untimestamped drops should explicitly choose to flush the mdb_catalog writes after committing. However, this solution appears to be risky for a couple of reasons:&lt;/p&gt;

&lt;p&gt;1) In the future, engineers might easily overlook the need to enable the flush option.&lt;br/&gt;
2) It would be challenging to identify all the places in the codebase where untimestamped drops occur and then update those places to choose the flush option.&lt;/p&gt;

&lt;p&gt;Therefore, I believe we should consider a more general solution, such as having the KV ident dropper perform an unstable checkpoint before dropping the ident with a drop timestamp as 0 (i.e., an untimestamped drop). (???)&lt;/p&gt;</comment>
                            <comment id="5679138" author="JIRAUSER1264163" created="Tue, 5 Sep 2023 11:03:16 +0000"  >&lt;p&gt;One solution to this could be to reorder operations such that we first write the catalog changes, flush those changes to disk, THEN perform the drop pending steps. In this manner the state of an ident being present on the catalog but not in WT becomes impossible.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="2439436">SERVER-80974</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="2439436">SERVER-80974</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="2449132">SERVER-81355</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="2460039">SERVER-81879</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="26008"><![CDATA[Storage Execution EMEA]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 5 Sep 2023 11:03:16 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        18 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>gregory.noma@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            18 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>gregory.wlodarek@mongodb.com</customfieldvalue>
            <customfieldvalue>jordi.olivares-provencio@mongodb.com</customfieldvalue>
            <customfieldvalue>suganthi.mani@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i2j5u7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i21afk:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="7684">Execution EMEA Team 2023-09-18</customfieldvalue>
    <customfieldvalue id="7685">Execution EMEA Team 2023-10-02</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i2irzj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>