<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:05:42 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-66529] The oplog manager thread updating the oplogReadTimestamp can race with a cappedTruncateAfter operation directly updating the oplogReadTimestamp</title>
                <link>https://jira.mongodb.org/browse/SERVER-66529</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;TL;DR the periodic thread updating oplogReadTimestamp doesn&apos;t have sufficient mutex coverage to avoid immediately setting the oplogReadTimestamp forward again after cappedTruncateAfter does a direct oplogReadTimestamp update backwards. If the timing is just right.&lt;/p&gt;

&lt;p&gt;&#160;-----------------------------------------------------------------&lt;br/&gt;
This is my explanation from the test failure ticket:&lt;/p&gt;

&lt;p&gt;RecordStore::cappedTruncateAfter has &lt;a href=&quot;https://github.com/mongodb/mongo/blob/3b24de4cdac250de6c74874d1410ab6d433d79dc/src/mongo/db/storage/wiredtiger/wiredtiger_record_store.cpp#L2115&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;special logic to update the oplogReadTimestamp&lt;/a&gt; if it&apos;s the record store for the oplog collection. Meanwhile, there&apos;s &lt;a href=&quot;https://github.com/mongodb/mongo/blob/3b24de4cdac250de6c74874d1410ab6d433d79dc/src/mongo/db/storage/wiredtiger/wiredtiger_oplog_manager.cpp#L193-L274&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;a thread that periodically updates the oplogReadTimestamp&lt;/a&gt;. Of note in the thread&apos;s logic, it &lt;a href=&quot;https://github.com/mongodb/mongo/blob/3b24de4cdac250de6c74874d1410ab6d433d79dc/src/mongo/db/storage/wiredtiger/wiredtiger_oplog_manager.cpp#L242-L259&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;releases the mutex protecting oplogReadTimestamp writes/reads while fetching the WT all_durable timestamp&lt;/a&gt;. So here&apos;s what I propose happened:&lt;/p&gt;

&lt;p&gt;1. The oplogReadTimestamp is T(5,30)&lt;br/&gt;
2. &lt;a href=&quot;https://github.com/mongodb/mongo/blob/3b24de4cdac250de6c74874d1410ab6d433d79dc/src/mongo/db/storage/wiredtiger/wiredtiger_oplog_manager.cpp#L246&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;PeriodicThread fetches the all_durable timestamp T(5,30)&lt;/a&gt;&lt;br/&gt;
3. &lt;a href=&quot;https://github.com/mongodb/mongo/blob/3b24de4cdac250de6c74874d1410ab6d433d79dc/src/mongo/db/storage/wiredtiger/wiredtiger_record_store.cpp#L2081-L2094&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Op1 truncates the oplog back to T(5,1), deleting T(5,20) &amp;amp; T(5,30)&lt;/a&gt;&lt;br/&gt;
4. &lt;a href=&quot;https://github.com/mongodb/mongo/blob/3b24de4cdac250de6c74874d1410ab6d433d79dc/src/mongo/db/storage/wiredtiger/wiredtiger_record_store.cpp#L2115&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Op1 then sets the oplogReadTimestamp to T(5,3)&lt;/a&gt;&lt;br/&gt;
5. &lt;a href=&quot;https://github.com/mongodb/mongo/blob/3b24de4cdac250de6c74874d1410ab6d433d79dc/src/mongo/db/storage/wiredtiger/wiredtiger_oplog_manager.cpp#L261-L264&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;PeriodicThread then moves the oplorReadTimestamp forward to T(5,30)&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&#160;So in theory, any internal operation truncating the oplog while the server is up and running (not startup or rollback) could cause this race. If such code exists anywhere. Startup and rollback both restart the storage engine, reseting the all_durable timestamp, and do not have this issue with oplog truncation.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2048880">SERVER-66529</key>
            <summary>The oplog manager thread updating the oplogReadTimestamp can race with a cappedTruncateAfter operation directly updating the oplogReadTimestamp</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="dianna.hohensee@mongodb.com">Dianna Hohensee</assignee>
                                    <reporter username="dianna.hohensee@mongodb.com">Dianna Hohensee</reporter>
                        <labels>
                    </labels>
                <created>Tue, 17 May 2022 18:58:29 +0000</created>
                <updated>Sun, 29 Oct 2023 21:38:09 +0000</updated>
                            <resolved>Tue, 14 Jun 2022 16:56:47 +0000</resolved>
                                                    <fixVersion>6.0.1</fixVersion>
                    <fixVersion>6.1.0-rc0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="4692479" author="xgen-internal-githook" created="Wed, 20 Jul 2022 16:21:36 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Dianna Hohensee&apos;, &apos;email&apos;: &apos;dianna.hohensee@mongodb.com&apos;, &apos;username&apos;: &apos;DiannaHohensee&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-66529&quot; title=&quot;The oplog manager thread updating the oplogReadTimestamp can race with a cappedTruncateAfter operation directly updating the oplogReadTimestamp&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-66529&quot;&gt;&lt;del&gt;SERVER-66529&lt;/del&gt;&lt;/a&gt; OplogManager must not release mutex, so as to avoid wrongly fowarding the oplogReadTimestamp after a cappedTruncateAfter operation sets it back.&lt;/p&gt;

&lt;p&gt;(cherry picked from commit d9e643fe4dfb7da4ab68d6929477613f244ad361)&lt;br/&gt;
Branch: v6.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/4e85654b613ab91f9507cd97997db5fa62c3076e&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/4e85654b613ab91f9507cd97997db5fa62c3076e&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4614799" author="xgen-internal-githook" created="Tue, 14 Jun 2022 15:35:28 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Dianna Hohensee&apos;, &apos;email&apos;: &apos;dianna.hohensee@mongodb.com&apos;, &apos;username&apos;: &apos;DiannaHohensee&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-66529&quot; title=&quot;The oplog manager thread updating the oplogReadTimestamp can race with a cappedTruncateAfter operation directly updating the oplogReadTimestamp&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-66529&quot;&gt;&lt;del&gt;SERVER-66529&lt;/del&gt;&lt;/a&gt; OplogManager must not release mutex, so as to avoid wrongly fowarding the oplogReadTimestamp after a cappedTruncateAfter operation sets it back.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/d9e643fe4dfb7da4ab68d6929477613f244ad361&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/d9e643fe4dfb7da4ab68d6929477613f244ad361&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4556665" author="dianna.hohensee" created="Wed, 18 May 2022 14:48:09 +0000"  >&lt;p&gt;We should test what happens to the WT all_durable timestamp when the oplog is truncated. I&apos;m not sure whether it unwinds, in which case the unit test isn&apos;t going to work as designed.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23470"><![CDATA[v6.0]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 10 Jun 2022 18:00:20 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 29 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-2841</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 29 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>dianna.hohensee@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0v8t3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr23x1:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="6030">Execution Team 2022-06-27</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0uuyf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>