<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:25:05 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-30809] Investigating remaining writes to the [KV]Catalog that must be timestamped.</title>
                <link>https://jira.mongodb.org/browse/SERVER-30809</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;The KVCatalog is a single table used by non-MMAP storage engines that backs all collection and index metadata (along with the &quot;Feature Tracker Document&quot; which need not be considered here).&lt;/p&gt;

&lt;p&gt;In the KVCatalog, each collection is a document and each index is embedded in its collection document. Currently, writes are only timestamped if there&apos;s an associated oplog entry. &lt;/p&gt;

&lt;p&gt;Creating an index makes two writes to the catalog. The first is written before the index build starts, to insert the index entry into the catalog with a key/value of &lt;tt&gt;ready: false&lt;/tt&gt; that is not replicated. The second write occurs when the index build completes. &lt;tt&gt;ready&lt;/tt&gt; is set to &lt;tt&gt;true&lt;/tt&gt; and an oplog entry is replicated. The first write is not timestamped while the second write is.&lt;/p&gt;

&lt;p&gt;Consider the following sequence:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;The stable timestamp is 1. Collection &quot;foo&quot; is in some initial state.&lt;/li&gt;
	&lt;li&gt;Time 2: &quot;foo&quot; gets a new &quot;validation schema&quot; (persisted in foo&apos;s document in KVCatalog).&lt;/li&gt;
	&lt;li&gt;Time 3: A new index build starts on &quot;foo&quot;. This update is not timestamped.&lt;/li&gt;
	&lt;li&gt;&lt;tt&gt;rollback_to_stable&lt;/tt&gt; is called on WiredTiger.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;The desired state for &quot;foo&quot; is in the initial state before the updates at time 2 and 3. However, &lt;tt&gt;rollback_to_stable&lt;/tt&gt; will restore the document to its &quot;Time 3&quot; state.&lt;/p&gt;

&lt;p&gt;My understanding is it should be legal for &lt;tt&gt;ready: false&lt;/tt&gt; index writes to have any timestamp &amp;gt;= to the last write on that document in the KVCatalog and &amp;lt; all futures to that document on the KVCatalog.&lt;/p&gt;</description>
                <environment></environment>
        <key id="419727">SERVER-30809</key>
            <summary>Investigating remaining writes to the [KV]Catalog that must be timestamped.</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="backlog-server-repl">Backlog - Replication Team</assignee>
                                    <reporter username="daniel.gottlieb@mongodb.com">Daniel Gottlieb</reporter>
                        <labels>
                    </labels>
                <created>Thu, 24 Aug 2017 03:11:57 +0000</created>
                <updated>Tue, 6 Dec 2022 03:53:43 +0000</updated>
                            <resolved>Wed, 20 Dec 2017 20:21:13 +0000</resolved>
                                                                    <component>Replication</component>
                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="1757173" author="daniel.gottlieb@10gen.com" created="Wed, 20 Dec 2017 20:20:40 +0000"  >&lt;p&gt;This ticket resulted in an investigation where an assertion was added to ensure all updates to the [KV]Catalog that may be rolled back rollback have a timestamp. Provisional changes were made to incrementally uncover all code paths evergreen exercises that result in writes to the KVCatalog for primaries and secondaries. &lt;/p&gt;

&lt;p&gt;The individual work items that investigation surfaced are linked to this ticket. In summary the remaining writes to timestamp are:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Commands coming into secondaries&lt;/li&gt;
	&lt;li&gt;The beginning of index builds on primaries and the completion of index builds on secondaries.&lt;/li&gt;
	&lt;li&gt;The drop phase of two phase drops on primaries and secondaries. On primaries, the drop phase for a collection can be performed by the reaper, or by a &lt;tt&gt;dropDatabase&lt;/tt&gt; command.&lt;/li&gt;
	&lt;li&gt;Setting &lt;tt&gt;multikey[Paths]&lt;/tt&gt; to true.&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="1657742" author="daniel.gottlieb@10gen.com" created="Fri, 25 Aug 2017 23:44:45 +0000"  >&lt;p&gt;I think readConcern: majority against the oplog should work as expected.&lt;/p&gt;</comment>
                            <comment id="1657655" author="spencer" created="Fri, 25 Aug 2017 21:11:46 +0000"  >&lt;p&gt;Does this mean it will no longer be possible to do a readConcern:majority read against the oplog and see the state of the oplog as of the replication commit point?&lt;/p&gt;</comment>
                            <comment id="1657366" author="daniel.gottlieb@10gen.com" created="Fri, 25 Aug 2017 17:14:54 +0000"  >&lt;p&gt;Addendum: Writes to the `_mdb_catalog` on behalf of a collection that&apos;s logged (e.g: the oplog or user collections in `local`), unfortunately need to survive a call to `rollback_to_stable`.&lt;/p&gt;

&lt;p&gt;The conjectured fix: given each (non-feature) document in the `_mdb_catalog` represents a single collection, a namespace should always be available when updating the catalog. The namespace can be passed to `WiredTigerUtil::useTableLogging` (which, for aesthetics, would presumably be pulled up from the WT layer to the KV layer) to determine whether a timestamp should be applied to the write. Writes for a logged table should not be timestamped.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="468757">SERVER-32185</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="468774">SERVER-32187</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="469311">SERVER-32206</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="470428">SERVER-32251</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="422433">SERVER-30911</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="468804">SERVER-32188</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25128"><![CDATA[Replication]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 25 Aug 2017 21:11:46 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 8 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-32185'>SERVER-32185</a></s>, <s><a href='https://jira.mongodb.org/browse/SERVER-32187'>SERVER-32187</a></s>, <s><a href='https://jira.mongodb.org/browse/SERVER-32206'>SERVER-32206</a></s>, <s><a href='https://jira.mongodb.org/browse/SERVER-32251'>SERVER-32251</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-842</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 8 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-repl</customfieldvalue>
            <customfieldvalue>daniel.gottlieb@mongodb.com</customfieldvalue>
            <customfieldvalue>spencer@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htdjmn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htbo27:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2031">Repl 2017-12-18</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htd5pj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>