<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:36:22 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-55397] Index build restart ident drops are not timestamped during startup recovery</title>
                <link>https://jira.mongodb.org/browse/SERVER-55397</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/7951ef4933ba29630e5407caf56e07569ab9f4ea/src/mongo/db/storage/storage_engine_impl.cpp#L593-L601&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;This assertion&lt;/a&gt; added in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-38910&quot; title=&quot;Remove redundant rollback handling on index drops&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-38910&quot;&gt;&lt;del&gt;SERVER-38910&lt;/del&gt;&lt;/a&gt;, made an assumption that all ident drops are timestamped. During startup recovery, if an index build is unfinished and needs to be restarted, we drop the original ident without a timestamp. This can cause this assertion to fail in certain circumstances.&lt;/p&gt;

&lt;p&gt;I think we need to either:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Timestamp the ident drop on startup recovery. This would ensure we don&apos;t drop the ident until it is majority-committed. Unfortunately, we aren&apos;t starting the index build from oplog entry, so we would potentially have to use a timestamp in the future, ahead of the recovery timestamp.&lt;/li&gt;
	&lt;li&gt;Relax this assertion to exclude unfinished index builds. Similar to the original code, if we discover that an ident is missing for an index, we only need to guarantee it will be rebuilt. If we do this, we also need to ensure that we don&apos;t error if we try to drop an ident that doesn&apos;t exist.&lt;/li&gt;
&lt;/ul&gt;
</description>
                <environment></environment>
        <key id="1655268">SERVER-55397</key>
            <summary>Index build restart ident drops are not timestamped during startup recovery</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="gregory.noma@mongodb.com">Gregory Noma</assignee>
                                    <reporter username="louis.williams@mongodb.com">Louis Williams</reporter>
                        <labels>
                    </labels>
                <created>Mon, 22 Mar 2021 14:40:42 +0000</created>
                <updated>Sun, 29 Oct 2023 21:55:57 +0000</updated>
                            <resolved>Wed, 5 May 2021 22:15:45 +0000</resolved>
                                    <version>4.9.0</version>
                                    <fixVersion>5.0.0-rc0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="3757741" author="xgen-internal-githook" created="Wed, 5 May 2021 22:08:22 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Gregory Noma&apos;, &apos;email&apos;: &apos;gregory.noma@gmail.com&apos;, &apos;username&apos;: &apos;gregorynoma&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-55397&quot; title=&quot;Index build restart ident drops are not timestamped during startup recovery&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-55397&quot;&gt;&lt;del&gt;SERVER-55397&lt;/del&gt;&lt;/a&gt; Relax index ident reconciliation invariant when restarting an index build after an unclean shutdown&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/95bf4b5d5d1c88002452cf8bd1e7ffcc47b68d9a&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/95bf4b5d5d1c88002452cf8bd1e7ffcc47b68d9a&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3703596" author="louis.williams" created="Tue, 6 Apr 2021 17:21:35 +0000"  >&lt;p&gt;Update: It turns out that the reason the index is dropped and restarted at all is that the rollback fuzzer &lt;a href=&quot;https://github.com/mongodb/mongo/blob/0d45dd9d7ba9d3a1557217a998ad31c68a897d47/etc/evergreen.yml#L4578-L4591&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;disables commitQuorum for index builds&lt;/a&gt;. As a result, the index build &lt;a href=&quot;https://github.com/mongodb/mongo/blob/0d45dd9d7ba9d3a1557217a998ad31c68a897d47/src/mongo/db/index_builds_coordinator.cpp#L425-L427&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;does not qualify as &quot;resumable&quot;&lt;/a&gt;, requiring the ident to be dropped and recreated at startup.&lt;/p&gt;

&lt;p&gt;Since index builds are resumable from every phase in 4.4, both of these bug conditions only exist when the server is configured with &lt;tt&gt;enableIndexBuildCommitQuorum=false&lt;/tt&gt;. As far as I know, this is not a parameter that we document.&lt;/p&gt;

&lt;p&gt; &lt;del&gt;I can confidently say that this is not required for 5.0.&lt;/del&gt; Users may, however, disable resumable index builds by providing a commitQuorum &lt;a href=&quot;https://github.com/mongodb/mongo/blob/8222bef16ae26297f7313274e4ca15eb1dd54655/src/mongo/db/index_builds_coordinator.cpp#L406-L431&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;other than &quot;votingMembers&quot;&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="3701643" author="louis.williams" created="Mon, 5 Apr 2021 19:52:28 +0000"  >&lt;p&gt;Considering this, I think there is probably a different bug, as follows:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;During startup recovery, there is an unfinished two-phase index build. We drop the ident and perform an un-timestamped write to the catalog entry to restart the index build.&lt;/li&gt;
	&lt;li&gt;Before replicating a &lt;tt&gt;commitIndexBuild&lt;/tt&gt; oplog entry, take a checkpoint and shut down. This persists the new catalog entry and the wiped-out history (the effect of the un-timestamped write) for that catalog entry.&lt;/li&gt;
	&lt;li&gt;The server restarts. There is no fatal assertion because the catalog entry&apos;s index matches an existing index table. The catalog entry shows incorrect history for all timestamped reads between the oldest and stable timestamps.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;If this were just a single startup (i.e. just step 1) this would not be possible, because we &lt;a href=&quot;https://github.com/mongodb/mongo/blob/9cfedfbac73d1783dae0d54343e4a0f8f6a5194c/src/mongo/db/catalog/catalog_control.cpp#L215&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;set collection&apos;s minimumVisibleSnapshot to the stable timestamp&lt;/a&gt; when the catalog entry is not visible at the oldest timestamp. But in this scenario, the catalog entry would be visible at both the stable timestamp and the oldest timestamp. As a result, we don&apos;t set the minimumVisible, allowing reads to observe a potentially incorrect historical state of the catalog entry.&lt;/p&gt;</comment>
                            <comment id="3698651" author="daniel.gottlieb@10gen.com" created="Fri, 2 Apr 2021 15:00:06 +0000"  >&lt;blockquote&gt;
&lt;p&gt;Daniel Gottlieb, when we do the drop of the index entry, this happens inside of one WUOW, so there is never a state without an index entry.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;To clarify, I was claiming that performing a write to an _mdb_catalog entry (for a replicated collection) without a timestamp can corrupt the update chain. &lt;/p&gt;

&lt;p&gt;Prior to durable history, a case like this would be &quot;okay&quot; so long as we haven&apos;t performed any replication recovery (meaning the 0 timestamped write is only covering up the version as of the stable checkpoint MDB is starting up against). As you demonstrated, that single WUOW just changes one index ident value for another.&lt;/p&gt;

&lt;p&gt;With durable history, we might be okay (modulo WT bugs coping with the complexity of this MDB requirement), but that would true only if we never read the _mdb_catalog at an arbitrarily early value (presumably between the oldest and stable timestamps).&lt;/p&gt;</comment>
                            <comment id="3698549" author="louis.williams" created="Fri, 2 Apr 2021 14:40:17 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;, when we do the drop of the index entry, &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a36cc452e1e49305eb4f2505eae770b75fe08d5f/src/mongo/db/index_builds_coordinator.cpp#L535-L540&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this happens inside of one WUOW&lt;/a&gt;, so there is never a state without an index entry. &lt;/p&gt;

&lt;p&gt;I think we could try to pass a larger timestamp to the timestamp monitor so that the old ident is not dropped until the next checkpoint.&lt;/p&gt;</comment>
                            <comment id="3698472" author="daniel.gottlieb@10gen.com" created="Fri, 2 Apr 2021 14:15:31 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=louis.williams&quot; class=&quot;user-hover&quot; rel=&quot;louis.williams&quot;&gt;louis.williams&lt;/a&gt;, does &quot;dropping the ident without a timestamp&quot; mean removing it from the _mdb_catalog entry for that collection?&lt;/p&gt;

&lt;p&gt;For a replicated collection I would assume that to be a data corrupting write. Am I missing something?&lt;/p&gt;

&lt;p&gt;&lt;b&gt;edit&lt;/b&gt;&lt;br/&gt;
Or maybe &quot;timestamping the ident drop&quot; means just giving a non-zero value to the monitor that tracks for when to remove an index.&lt;/p&gt;

&lt;p&gt;As per what &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=gregory.wlodarek&quot; class=&quot;user-hover&quot; rel=&quot;gregory.wlodarek&quot;&gt;gregory.wlodarek&lt;/a&gt; found in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-55552&quot; title=&quot;Unreplicated collection idents can get dropped before the drop in the durable catalog becomes both checkpointed and older than the oldest timestamp&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-55552&quot;&gt;&lt;del&gt;SERVER-55552&lt;/del&gt;&lt;/a&gt;, it&apos;s never okay to pass a zero-timestamp to the ident dropping monitor. We should probably add an invariant there.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="1660353">SERVER-55552</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="668212">SERVER-38910</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="1701124">SERVER-56639</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1802637">SERVER-58159</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 2 Apr 2021 14:15:31 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 40 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 40 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>137.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>daniel.gottlieb@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>gregory.noma@mongodb.com</customfieldvalue>
            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hz0k3j:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr423r:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4653">Execution Team 2021-04-19</customfieldvalue>
    <customfieldvalue id="4654">Execution Team 2021-05-03</customfieldvalue>
    <customfieldvalue id="4828">Execution Team 2021-05-17</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hz06cn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>