<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:25:27 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-30926] Add timestamps to writes to minvalid document</title>
                <link>https://jira.mongodb.org/browse/SERVER-30926</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Writes to the minValid document need to be timestamped so that they don&apos;t persist after a call to recoverToStableTimestamp.  During secondary batch application there are 2 main writes to the minvalid collection that are relevant: one that sets the minValid to the end of the batch and another that sets the appliedThrough to the end of the batch.  Both those writes should be given the timestamp of the end of the batch, so that if we recover to a time before the batch the writes will be undone.&lt;/p&gt;</description>
                <environment></environment>
        <key id="422948">SERVER-30926</key>
            <summary>Add timestamps to writes to minvalid document</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="judah.schvimer@mongodb.com">Judah Schvimer</assignee>
                                    <reporter username="spencer@mongodb.com">Spencer Brody</reporter>
                        <labels>
                    </labels>
                <created>Fri, 1 Sep 2017 21:18:33 +0000</created>
                <updated>Mon, 30 Oct 2023 23:13:52 +0000</updated>
                            <resolved>Wed, 13 Dec 2017 17:51:47 +0000</resolved>
                                                    <fixVersion>3.7.1</fixVersion>
                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="1751114" author="xgen-internal-githook" created="Wed, 13 Dec 2017 17:49:29 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Judah Schvimer&apos;, &apos;email&apos;: &apos;judah@mongodb.com&apos;, &apos;username&apos;: &apos;judahschvimer&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-30926&quot; title=&quot;Add timestamps to writes to minvalid document&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-30926&quot;&gt;&lt;del&gt;SERVER-30926&lt;/del&gt;&lt;/a&gt; Add timestamps to writes to minValid document&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/6796859387ba77a3556ed583a317681a288970e4&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/6796859387ba77a3556ed583a317681a288970e4&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1743630" author="judah.schvimer" created="Tue, 5 Dec 2017 22:40:53 +0000"  >&lt;p&gt;There are 7 types of writes to the minValid document to consider:&lt;br/&gt;
1. &lt;b&gt;minValid updates after we write a batch of oplog entries before we apply them:&lt;/b&gt; These we will timestamp with the minValid time we are writing. We only take stable checkpoints when we are consistent. Thus, the next checkpoint we will take is at this minValid. If we gave it a timestamp from before the batch, and we took a stable checkpoint at that timestamp, then we would consider that timestamp inconsistent, even though it is.&lt;br/&gt;
2. &lt;b&gt;minValid updates during rollback via refetch:&lt;/b&gt; These updates should only occur on storage engines that do not support recover to stable timestamp, and thus the timestamp should not matter. We will give them a 0 timestamp and add an invariant that we are using a storage engine that does not support recover to stable timestamp.&lt;br/&gt;
3. &lt;b&gt;minValid initialization&lt;/b&gt;: This occurs at startup, at initiate, and on secondaries when they receive their first config. We will give these a timestamp of 0 since we want them to be in the first checkpoint, even if the checkpoint is for a timestamp in the past. The minValid document could exist already and this could simply add fields to the minValid document, but we still want the initialization write to go into the next checkpoint since a newly initialized minValid document is always valid.&lt;br/&gt;
4. &lt;b&gt;removing the old oplog delete from point:&lt;/b&gt; This field is going to be removed in 3.8 in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-30556&quot; title=&quot;remove old OplogDeleteFromPoint from minValid collection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-30556&quot;&gt;&lt;del&gt;SERVER-30556&lt;/del&gt;&lt;/a&gt;, so we do not care about the write. We will give it a timestamp of 0 in the meantime.&lt;br/&gt;
5. &lt;b&gt;setting appliedThrough:&lt;/b&gt; This occurs in many places. &lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;The first is when we first establish a sync source. This sets it to the last applied optime, and should get that same timestamp.&lt;/li&gt;
	&lt;li&gt;The next is rollback via refetch which clears appliedThrough so we check the top of the oplog for the appliedThrough. These updates should only occur on storage engines that do not support recover to stable timestamp, and thus the timestamp should not matter. We will give them a 0 timestamp and add an invariant that we are using a storage engine that does not support recover to stable timestamp.&lt;/li&gt;
	&lt;li&gt;The next is SyncTail after we&apos;ve applied a batch of oplog entries. This should set it to the same timestamp since that&apos;s where the data is at.&lt;/li&gt;
	&lt;li&gt;It is cleared at shutdown to indicate we&apos;re consistent at the top of the oplog. This should get the last applied optime for the timestamp in case we&apos;re in the process of taking a checkpoint at an earlier timestamp and do not want that checkpoint to reflect this write.&lt;/li&gt;
	&lt;li&gt;It is also cleared when transitioning to primary to indicate we&apos;re consistent at the top of the oplog. This should get the last applied optime for the timestamp so no checkpoints at earlier timestamps get this write.&lt;/li&gt;
	&lt;li&gt;It is set during recovery after each oplog entry is applied. This can get the optime from the oplog entry like in the 3rd bullet.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;6. &lt;b&gt;Setting the initial sync flag at the beginning of initial sync:&lt;/b&gt; This will get a 0 timestamp because it will be in no stable checkpoints.&lt;br/&gt;
7. &lt;b&gt;Clearing the initial sync flag at the end of initial sync:&lt;/b&gt; This will get the last applied optime as the timestamp for clarity, though there cannot be any checkpoints taken before it, so it could be 0 as well.&lt;/p&gt;

&lt;p&gt;CC &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1737629" author="judah.schvimer" created="Wed, 29 Nov 2017 18:29:08 +0000"  >&lt;p&gt;These writes do need to be timestamped since the checkpoint thread runs asynchronously and the minValid document could change between the time at which the checkpoint is taken and the time that it is read for the checkpoint.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="411399">SERVER-30472</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 29 Nov 2017 18:29:08 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 9 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-842</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 9 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>
            <customfieldvalue>spencer@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hte3ev:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|ht4qjb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="1957">Repl 2017-12-04</customfieldvalue>
    <customfieldvalue id="2031">Repl 2017-12-18</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htdphr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>