<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:07:37 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-45010] Clean shutdown after rollbackViaRefetch with eMRC=false can cause us to incorrectly overwrite unstable checkpoints</title>
                <link>https://jira.mongodb.org/browse/SERVER-45010</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;At the end of rollbackViaRefetch when eMRC=false, we &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c45ad5a6a54f04270a6abc40d0d71104b8f9c21b/src/mongo/db/repl/rs_rollback.cpp#L1565&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;will take an unstable checkpoint&lt;/a&gt; before proceeding. After this is complete, we enter RECOVERING state and try to catch up our oplog. If the server is shut down cleanly after we take these unstable checkpoints, though, upon shutdown we will explicitly take another checkpoint, &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c45ad5a6a54f04270a6abc40d0d71104b8f9c21b/src/third_party/wiredtiger/src/include/wiredtiger.in#L2084-L2085&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;which will be &lt;em&gt;stable&lt;/em&gt;&lt;/a&gt; because we have a stable timestamp set. We do &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c45ad5a6a54f04270a6abc40d0d71104b8f9c21b/src/mongo/db/repl/rs_rollback.cpp#L1505-L1520&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;set the initialDataTimestamp&lt;/a&gt; ahead of the stable timestamp after rollbackViaRefetch so that no checkpoints are taken, but the logic that normally handles that &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c45ad5a6a54f04270a6abc40d0d71104b8f9c21b/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp#L413-L459&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;in the WTCheckpointThread&lt;/a&gt; is bypassed during shutdown, so we &lt;a href=&quot;https://github.com/mongodb/mongo/blob/f4398b4afbaeaa388f6a7360949af006c632df07/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp#L969&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;take a full stable checkpoint&lt;/a&gt; regardless of the initialDataTimestamp value. Taking a stable checkpoint and recovering from it on restart in this case causes us to break the assumptions required for the correctness of rollbackViaRefetch with eMRC=false. See  &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-38925&quot; title=&quot;Rollback via refetch can cause _id duplication when enableMajorityReadConcern:false&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-38925&quot;&gt;&lt;del&gt;SERVER-38925&lt;/del&gt;&lt;/a&gt; for an explanation of why these unstable checkpoints are necessary.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1040848">SERVER-45010</key>
            <summary>Clean shutdown after rollbackViaRefetch with eMRC=false can cause us to incorrectly overwrite unstable checkpoints</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="william.schultz@mongodb.com">William Schultz</assignee>
                                    <reporter username="william.schultz@mongodb.com">William Schultz</reporter>
                        <labels>
                            <label>KP42</label>
                    </labels>
                <created>Fri, 6 Dec 2019 22:59:35 +0000</created>
                <updated>Sun, 29 Oct 2023 22:14:18 +0000</updated>
                            <resolved>Wed, 22 Jan 2020 22:59:25 +0000</resolved>
                                    <version>4.2.1</version>
                    <version>4.3.2</version>
                                    <fixVersion>4.2.4</fixVersion>
                    <fixVersion>4.3.3</fixVersion>
                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>10</watches>
                                                                                                                <comments>
                            <comment id="3076597" author="william.schultz" created="Fri, 8 May 2020 20:44:34 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=henrik.edin&quot; class=&quot;user-hover&quot; rel=&quot;henrik.edin&quot;&gt;henrik.edin&lt;/a&gt; That commit originally appeared in 4.3.3, so it predated v4.4.&lt;/p&gt;</comment>
                            <comment id="3021307" author="william.schultz" created="Wed, 1 Apr 2020 16:10:29 +0000"  >&lt;p&gt;Adding a note here for future reference about why we can&apos;t take &lt;em&gt;unstable&lt;/em&gt; checkpoints on a clean shutdown after rollback. We don&#8217;t want to take stable checkpoints after rollback until we &lt;a href=&quot;https://github.com/mongodb/mongo/blob/59152df7655101ee551d6432727170b42fcd136f/src/mongo/db/repl/rs_rollback.cpp#L1724-L1733&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;reach&lt;/a&gt; max(localTopOfOplog, syncSourceTopOfOplog), and we take an unstable checkpoint before leaving rollback to include all writes performed during the rollback. We also &lt;a href=&quot;https://github.com/mongodb/mongo/blob/edb8778350326d2b33f056b1b5f0b25a4b5b444a/src/mongo/db/repl/rs_rollback.cpp#L1978-L1990&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;set appliedThrough to the common point&lt;/a&gt; so that if we crash and restart before the first stable checkpoint we replay oplog entries from the common point for recovery. The question is whether it is safe to take additional &lt;em&gt;unstable&lt;/em&gt; checkpoints before we reach max(localTopOfOplog, syncSourceTopOfOplog). If we did this and restarted, we would replay oplog entries from the common point, potentially replaying ones we already applied, which could encounter oplog idempotency issues. Initial sync is allowed to fail if it encounters an idempotency issues, but startup recovery must not. So, we can&apos;t take unstable checkpoints on shutdown either in order to avoid idempotency issues on recovery.&lt;/p&gt;
</comment>
                            <comment id="2861589" author="xgen-internal-githook" created="Tue, 11 Feb 2020 03:19:49 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;William Schultz&apos;, &apos;username&apos;: &apos;will62794&apos;, &apos;email&apos;: &apos;william.schultz@mongodb.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-45010&quot; title=&quot;Clean shutdown after rollbackViaRefetch with eMRC=false can cause us to incorrectly overwrite unstable checkpoints&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-45010&quot;&gt;&lt;del&gt;SERVER-45010&lt;/del&gt;&lt;/a&gt; Avoid taking a checkpoint on clean shutdown if stableTimestamp &amp;lt; initialDataTimestamp&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 759e930c88081aa0fb86e34a3ce7b2ed190c806e)&lt;/p&gt;

&lt;p&gt; create mode 100644 jstests/replsets/rollback_dup_ids_clean_shutdown_during_rollback.js&lt;br/&gt;
Branch: v4.2&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/873a18d34703e037f52162af693dcdc5901cb28c&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/873a18d34703e037f52162af693dcdc5901cb28c&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2754028" author="xgen-internal-githook" created="Wed, 22 Jan 2020 22:54:53 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;email&apos;: &apos;william.schultz@mongodb.com&apos;, &apos;username&apos;: &apos;will62794&apos;, &apos;name&apos;: &apos;William Schultz&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-45010&quot; title=&quot;Clean shutdown after rollbackViaRefetch with eMRC=false can cause us to incorrectly overwrite unstable checkpoints&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-45010&quot;&gt;&lt;del&gt;SERVER-45010&lt;/del&gt;&lt;/a&gt; Avoid taking a checkpoint on clean shutdown if stableTimestamp &amp;lt; initialDataTimestamp&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/759e930c88081aa0fb86e34a3ce7b2ed190c806e&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/759e930c88081aa0fb86e34a3ce7b2ed190c806e&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2748201" author="william.schultz" created="Tue, 21 Jan 2020 20:14:21 +0000"  >&lt;p&gt;The WiredTiger &lt;a href=&quot;https://github.com/mongodb/mongo/blob/8e2c33cdf3552a7ba3a96fef162a3463f16a33eb/src/third_party/wiredtiger/src/txn/txn.c#L1556-L1585&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;checkpoint logic&lt;/a&gt; at shutdown inside &lt;tt&gt;__wt_txn_global_shutdown&lt;/tt&gt;.&lt;/p&gt;</comment>
                            <comment id="2715214" author="tess.avitabile" created="Thu, 9 Jan 2020 20:53:39 +0000"  >&lt;p&gt;Great! Thank you for letting me know.&lt;/p&gt;</comment>
                            <comment id="2714917" author="william.schultz" created="Thu, 9 Jan 2020 19:11:34 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=steven.vannelli&quot; class=&quot;user-hover&quot; rel=&quot;steven.vannelli&quot;&gt;steven.vannelli&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tess.avitabile%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;tess.avitabile@mongodb.com&quot;&gt;tess.avitabile@mongodb.com&lt;/a&gt; I have a proper repro written and a patch in progress so am fairly close to being ready for a code review. &lt;/p&gt;</comment>
                            <comment id="2590892" author="william.schultz" created="Fri, 6 Dec 2019 23:00:36 +0000"  >&lt;p&gt;Applying the attached diff (to base commit bb7120c13) and running &lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;python3 buildscripts/resmoke.py --majorityReadConcern=off  jstests/replsets/rollback_dup_ids.js&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;should reproduce the issue. I have also reproduced this on the 4.2 branch.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="1253193">SERVER-46714</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="669232">SERVER-38925</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="1299490">SERVER-47219</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1107641">WT-5466</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="239667" name="bf-15306-repro.diff" size="3069" author="william.schultz@mongodb.com" created="Fri, 6 Dec 2019 22:58:45 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16775"><![CDATA[v4.2]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 9 Jan 2020 19:08:45 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 39 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 39 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>50.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>tess.avitabile@mongodb.com</customfieldvalue>
            <customfieldvalue>william.schultz@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hw8uan:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hvx2l3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="3438">Repl 2019-12-30</customfieldvalue>
    <customfieldvalue id="3439">Repl 2020-01-13</customfieldvalue>
    <customfieldvalue id="3440">Repl 2020-01-27</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hw8gjz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>