<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:52:47 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-39679] Add callback to replication when storage takes a checkpoint to learn of the maximum oplog truncation timestamp</title>
                <link>https://jira.mongodb.org/browse/SERVER-39679</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Due to Prepare Support for Transactions and Larger Transactions Than 16MB, we cannot truncate transaction oplog entries if their commit oplog entry isn&apos;t stable yet.&lt;/p&gt;

&lt;p&gt;The current solution is to pass the current oldest active transaction to &lt;tt&gt;setStableTimestamp()&lt;/tt&gt;, so that we have to read the oldest active transaction at the stable timestamp, the &lt;em&gt;oldest required timestamp&lt;/em&gt;, every time we set the stable timestamp. To save this read, we could let the storage layer call back the replication system when it&apos;s about to start a checkpoint. Replication will read the &lt;em&gt;oldest required timestamp&lt;/em&gt; or calculate this timestamp in other ways, then return the timestamp to storage. After the checkpoint, storage uses the &lt;em&gt;oldest required timestamp&lt;/em&gt; to let oplog truncation thread know where it can truncate up to.&lt;/p&gt;

&lt;p&gt;Passing the &lt;em&gt;oldest required timestamp&lt;/em&gt; to storage can be done asynchronously if that makes storage work easier.&lt;/p&gt;</description>
                <environment></environment>
        <key id="700955">SERVER-39679</key>
            <summary>Add callback to replication when storage takes a checkpoint to learn of the maximum oplog truncation timestamp</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="jesse@mongodb.com">A. Jesse Jiryu Davis</assignee>
                                    <reporter username="siyuan.zhou@mongodb.com">Siyuan Zhou</reporter>
                        <labels>
                            <label>prepare_durability</label>
                    </labels>
                <created>Tue, 19 Feb 2019 22:58:47 +0000</created>
                <updated>Sun, 29 Oct 2023 22:23:51 +0000</updated>
                            <resolved>Fri, 22 Mar 2019 01:26:25 +0000</resolved>
                                                    <fixVersion>4.1.10</fixVersion>
                                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="2188222" author="xgen-internal-githook" created="Fri, 22 Mar 2019 01:24:08 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;A. Jesse Jiryu Davis&apos;, &apos;username&apos;: &apos;ajdavis&apos;, &apos;email&apos;: &apos;jesse@mongodb.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-39679&quot; title=&quot;Add callback to replication when storage takes a checkpoint to learn of the maximum oplog truncation timestamp&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-39679&quot;&gt;&lt;del&gt;SERVER-39679&lt;/del&gt;&lt;/a&gt; Get oldest transaction time when snapshotting&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/78eaa3cf538764d5aa5a09c5997532a4c3b2bca8&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/78eaa3cf538764d5aa5a09c5997532a4c3b2bca8&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2160232" author="judah.schvimer" created="Fri, 22 Feb 2019 19:57:30 +0000"  >&lt;blockquote&gt;
&lt;p&gt;The more &quot;design&quot;y change that&apos;s part of this ticket is to figure out how replication passes knowledge of the callback through the StorageEngine interface -&amp;gt; KVStorageEngine -&amp;gt; KVEngine -&amp;gt; WiredTigerKVEngine.&lt;/p&gt;

&lt;p&gt;The &quot;edge case&quot;y parts have to do with startup as storage naturally starts its checkpoint and oplog truncation threads before replication has a chance to register a callback.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;, we need this a little more urgently now that we&apos;ve pivoted our design for &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36494&quot; title=&quot;Prevent oplog truncation of oplog entries needed for startup recovery&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-36494&quot;&gt;&lt;del&gt;SERVER-36494&lt;/del&gt;&lt;/a&gt; to require this ticket. Can you please provide guidance on the above two pieces so we can take the implementation of this off of your hands without having too much come up in review?&lt;/p&gt;</comment>
                            <comment id="2159147" author="siyuan.zhou@10gen.com" created="Thu, 21 Feb 2019 22:14:08 +0000"  >&lt;p&gt;Sounds great. Thanks Dan! Storage can still cache &lt;tt&gt;_oplogNeededForRollback&lt;/tt&gt; in memory, so that it can report to serverStatus and FTDC.&lt;/p&gt;</comment>
                            <comment id="2156555" author="daniel.gottlieb@10gen.com" created="Wed, 20 Feb 2019 01:04:08 +0000"  >&lt;p&gt;The description makes sense to me. I&apos;ll highlight the what I think the code changes are (this is a dark corner of the storage code).&lt;/p&gt;

&lt;p&gt;I expect the only &quot;mechanical&quot; change is to have &lt;a href=&quot;https://github.com/mongodb/mongo/blob/8d18f3593747f5ef5ca6f40cf37014de117e1c9d/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp#L1866-L1868&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this method&lt;/a&gt; return the value of the callback, instead of reading a variable from memory.&lt;/p&gt;

&lt;p&gt;The more &quot;design&quot;y change that&apos;s part of this ticket is to figure out how replication passes knowledge of the callback through the StorageEngine interface -&amp;gt; KVStorageEngine -&amp;gt; KVEngine -&amp;gt; WiredTigerKVEngine.&lt;/p&gt;

&lt;p&gt;The &quot;edge case&quot;y parts have to do with startup as storage naturally starts its checkpoint and oplog truncation threads before replication has a chance to register a callback.&lt;/p&gt;</comment>
                            <comment id="2156464" author="siyuan.zhou@10gen.com" created="Tue, 19 Feb 2019 23:05:01 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;, does the description look accurate to you?&lt;/p&gt;</comment>
                            <comment id="2156460" author="judah.schvimer" created="Tue, 19 Feb 2019 23:02:38 +0000"  >&lt;p&gt;We expect this is required for good performance in 4.2, but can use the existing plan from &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36811&quot; title=&quot;Provide a mechanism for replication to specify the &amp;#39;maximum_truncation_timestamp&amp;#39; for a given &amp;#39;stable_timestamp&amp;#39;&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-36811&quot;&gt;&lt;del&gt;SERVER-36811&lt;/del&gt;&lt;/a&gt; in the meantime to get to correct behavior.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="584990">SERVER-36494</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="1804612">SERVER-58184</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="591247">SERVER-36772</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="700970">SERVER-39680</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="593010">SERVER-36811</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 19 Feb 2019 23:02:38 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 46 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1032</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 46 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>jesse@mongodb.com</customfieldvalue>
            <customfieldvalue>daniel.gottlieb@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>
            <customfieldvalue>siyuan.zhou@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huo3pj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hudupr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2822">Repl 2019-03-11</customfieldvalue>
    <customfieldvalue id="2823">Repl 2019-03-25</customfieldvalue>
    <customfieldvalue id="2896">Repl 2019-04-08</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hunpyv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>