<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:49:06 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-82396] PreImagesTruncateManager truncate initialization reads at lastApplied can miss inserts</title>
                <link>https://jira.mongodb.org/browse/SERVER-82396</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;By default, internal secondary readers &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-79955?focusedCommentId=5711883&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-5711883&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;read at &apos;lastApplied&apos;&lt;/a&gt;. By reading at &apos;lastApplied&apos; by default, the tenant&apos;s truncate marker initialization can miss a pre-image insert if initialization completes in the middle of oplog batch application on the secondary.&lt;/p&gt;

&lt;p&gt;The consequence: If an inserted pre-image has a higher RecordId that the highest RecordId tracked by the truncate markers, the pre-image won&apos;t be removed until the next insert.&lt;/p&gt;

&lt;p&gt;For example, suppose:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;The &apos;lastApplied&apos; timestamp is at TS(5), captured in snapshot0.&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/3d2361ddefbe9cda5e1fd5626bafde68fea4df57/src/mongo/db/change_stream_pre_images_truncate_manager.cpp#L681&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Truncate marker generation begins,&lt;/a&gt;&#160;sees 4 documents with nsUUID0 at snapshot0.&lt;/li&gt;
	&lt;li&gt;Secondary oplog batch application&#160;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/3d2361ddefbe9cda5e1fd5626bafde68fea4df57/src/mongo/db/change_stream_pre_images_truncate_manager.cpp#L667&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;inserts&lt;/a&gt;&#160;a 5th pre-image to nsUUID0 at TS(10), but oplog application is still between batches and lastApplied hasn&apos;t been advanced.
	&lt;ul&gt;
		&lt;li&gt;The insert updates the &apos;tenantMapEntry&apos;, serving as a&#160;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/3d2361ddefbe9cda5e1fd5626bafde68fea4df57/src/mongo/db/change_stream_pre_images_truncate_manager.cpp#L677&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;placeholder&lt;/a&gt;&#160;until truncate marker generation is complete, with a new set of truncate markers for nsUUID0 that have&#160;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/7a45b7e1a3b0da94ec029c960e172dca0b01c786/src/mongo/db/storage/collection_truncate_markers.cpp#L543&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;_lastHighestRecordId&lt;/a&gt;&#160;encoding TS(10).&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;The &apos;tenantMapEntry&apos; placeholder truncate markers for nsUUID0&#160;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/3d2361ddefbe9cda5e1fd5626bafde68fea4df57/src/mongo/db/change_stream_pre_images_truncate_manager.cpp#L696-L703&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;are overwritten&lt;/a&gt;&#160;because generatedTruncateMarkers also had a set of truncate markers for nsUUID0.&lt;/li&gt;
	&lt;li&gt;Even after&#160;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/3d2361ddefbe9cda5e1fd5626bafde68fea4df57/src/mongo/db/change_stream_pre_images_truncate_manager.cpp#L710-L719&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;abandoning snapshot0&lt;/a&gt;, the new snapshot still reads at lastApplied TS(5) because the oplog batch application hasn&apos;t completed yet.&#160;&lt;/li&gt;
	&lt;li&gt;Thus, the truncate markers never track the 5th document inserted into the pre-images collection.&#160;
	&lt;ul&gt;
		&lt;li&gt;If the truncate markers for nsUUID0 don&apos;t include the 5th pre-image, the 5th pre-image won&apos;t be removed until another pre-image for nsUUID0 is inserted, and updates the&#160;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/7a45b7e1a3b0da94ec029c960e172dca0b01c786/src/mongo/db/storage/collection_truncate_markers.h#L379&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;_lastHighestRecordId.&lt;/a&gt;&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
&lt;/ul&gt;
</description>
                <environment></environment>
        <key id="2481663">SERVER-82396</key>
            <summary>PreImagesTruncateManager truncate initialization reads at lastApplied can miss inserts</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="haley.connelly@mongodb.com">Haley Connelly</assignee>
                                    <reporter username="haley.connelly@mongodb.com">Haley Connelly</reporter>
                        <labels>
                    </labels>
                <created>Tue, 24 Oct 2023 10:18:46 +0000</created>
                <updated>Fri, 27 Oct 2023 12:08:02 +0000</updated>
                            <resolved>Fri, 27 Oct 2023 12:08:02 +0000</resolved>
                                                    <fixVersion>7.2.0-rc0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="5814261" author="xgen-internal-githook" created="Fri, 27 Oct 2023 11:18:50 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;Haley Connelly&apos;, &apos;email&apos;: &apos;haley.connelly@mongodb.com&apos;, &apos;username&apos;: &apos;haleyConnelly&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-82396&quot; title=&quot;PreImagesTruncateManager truncate initialization reads at lastApplied can miss inserts&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-82396&quot;&gt;&lt;del&gt;SERVER-82396&lt;/del&gt;&lt;/a&gt; Make pre image truncate marker initialization use ReadSource::kNoTimestamp&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/f95e3a502ca42a94305ab561978fc91a6cc0e97a&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/f95e3a502ca42a94305ab561978fc91a6cc0e97a&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 25 Oct 2023 15:18:51 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        14 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-3092</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>haley.connelly@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            14 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>haley.connelly@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i2xb1z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i2f6g4:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="7723">Execution EMEA Team 2023-10-30</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i2wx7b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>