<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:23:45 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-73131] clarify cached partial transaction ops state in oplog applier when aborting a prepared transaction</title>
                <link>https://jira.mongodb.org/browse/SERVER-73131</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;TLDR: enforce expectations on &lt;a href=&quot;https://github.com/mongodb/mongo/blob/1947ddfa0a6418ea1aae5fa720ffb43aa4b3a4e1/src/mongo/db/repl/oplog_applier_impl.cpp#L692&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;partialTxnOps&lt;/a&gt; in &lt;a href=&quot;https://github.com/mongodb/mongo/blob/1947ddfa0a6418ea1aae5fa720ffb43aa4b3a4e1/src/mongo/db/repl/oplog_applier_impl.cpp#L685&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;OplogApplierImpl::_deriveOpsAndFillWriterVectors()&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The function &lt;a href=&quot;https://github.com/mongodb/mongo/blob/1947ddfa0a6418ea1aae5fa720ffb43aa4b3a4e1/src/mongo/db/repl/oplog_applier_impl.cpp#L685&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;OplogApplierImpl::_deriveOpsAndFillWriterVectors()&lt;/a&gt; is used during oplog application to extract embedded operations from transaction oplog entries (ie. applyOps) from the current batch of operations returned from the oplog batching/fetching logic. The &lt;a href=&quot;https://github.com/mongodb/mongo/blob/1947ddfa0a6418ea1aae5fa720ffb43aa4b3a4e1/src/mongo/db/repl/oplog_applier_impl.cpp#L692&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;partialTxnOps&lt;/a&gt; map caches operations for multi-oplog transactions within the current batch - partial transaction operations outside the current batch in the large transaction is retrieved from the oplog collection in &lt;a href=&quot;https://github.com/mongodb/mongo/blob/1947ddfa0a6418ea1aae5fa720ffb43aa4b3a4e1/src/mongo/db/repl/oplog_applier_impl.cpp#L114&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;_addOplogChainOpsToWriterVectors()&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The batch processing logic currently &lt;a href=&quot;https://github.com/mongodb/mongo/blob/1947ddfa0a6418ea1aae5fa720ffb43aa4b3a4e1/src/mongo/db/repl/oplog_applier_impl.cpp#L738&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;clears the cached partial transaction operations&lt;/a&gt; when it encounters an abort transaction oplog entry in the same batch. Per current rules around oplog batch boundaries, especially with regards to OplogBatch::mustProcessIndividually()|https://github.com/mongodb/mongo/blob/1947ddfa0a6418ea1aae5fa720ffb43aa4b3a4e1/src/mongo/db/repl/oplog_batcher.cpp#L132], an abort transaction oplog entry must be the only operation in any oplog batch.&lt;/p&gt;

&lt;p&gt;It would be nice to clarify the logic in the oplog applier, preferably with an invariant on the expected state of &lt;tt&gt;partialTxnOps&lt;/tt&gt;, when we attempt to process an abort for a prepared transaction.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2238464">SERVER-73131</key>
            <summary>clarify cached partial transaction ops state in oplog applier when aborting a prepared transaction</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="benety.goh@mongodb.com">Benety Goh</assignee>
                                    <reporter username="benety.goh@mongodb.com">Benety Goh</reporter>
                        <labels>
                    </labels>
                <created>Fri, 20 Jan 2023 14:27:54 +0000</created>
                <updated>Sun, 29 Oct 2023 21:27:42 +0000</updated>
                            <resolved>Mon, 23 Jan 2023 15:47:05 +0000</resolved>
                                                    <fixVersion>6.3.0-rc0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="5128276" author="xgen-internal-githook" created="Fri, 20 Jan 2023 21:16:14 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Benety Goh&apos;, &apos;email&apos;: &apos;benety@mongodb.com&apos;, &apos;username&apos;: &apos;benety&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-73131&quot; title=&quot;clarify cached partial transaction ops state in oplog applier when aborting a prepared transaction&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-73131&quot;&gt;&lt;del&gt;SERVER-73131&lt;/del&gt;&lt;/a&gt; add invariant for abortTransaction and cached partial transaction operations in the same batch&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/d4c9920963d4475624eb58fc62ca0ce61f1f0846&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/d4c9920963d4475624eb58fc62ca0ce61f1f0846&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5127974" author="xgen-internal-githook" created="Fri, 20 Jan 2023 19:46:28 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Benety Goh&apos;, &apos;email&apos;: &apos;benety@mongodb.com&apos;, &apos;username&apos;: &apos;benety&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-73131&quot; title=&quot;clarify cached partial transaction ops state in oplog applier when aborting a prepared transaction&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-73131&quot;&gt;&lt;del&gt;SERVER-73131&lt;/del&gt;&lt;/a&gt; add test for preparing and aborting a multi-oplog transaction in the same batch&lt;/p&gt;

&lt;p&gt;Note this unit test covers a scenario currently disallowed under current oplog batching rules.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/da442eb11a61021e06237f7f0f5d4f30bd97d825&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/da442eb11a61021e06237f7f0f5d4f30bd97d825&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5126965" author="jason.chan" created="Fri, 20 Jan 2023 14:59:02 +0000"  >&lt;p&gt;Copying from my conversation with Benety.&lt;/p&gt;

&lt;p&gt;From what I remember:&lt;br/&gt;
We will buffer the partialTxn ops in a memory buffer as part of secondary transaction until we either 1. apply a prepareTxn entry or 2. apply a terminal apply op (an apply op entry which indicates we should commit the unprepared txn).&lt;br/&gt;
 To answer your question, I don&apos;t think we should expect to ever have existing map entries in partialTxnOps when processing an abortTransaction entry at the step you are referencing because:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;if unprepared txns are aborted, they will not generate an oplog entry and will have been aborted on the primary. So the secondary shouldn&apos;t see an abortOplog entry for unprepared transactions at all.&lt;/li&gt;
	&lt;li&gt;If a prepared txn is aborted, the partialTxns list should have already been applied and cleared as part of the secondary applying the prepareTxn entry.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;For extra context, one can refer to the &lt;a href=&quot;https://docs.google.com/document/d/1n2Jg_Zceh6XOv6o51W0Q3sku3Btn_3luO7orq6SpERs/edit&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;design of packing oplog formats&lt;/a&gt; for large transactions.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="685737">SERVER-39438</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="768946">SERVER-41157</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="941756">SERVER-43651</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 20 Jan 2023 14:35:36 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 2 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-2780</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 2 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>benety.goh@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>jason.chan@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1riuv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i1a17c:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="6684">Execution Team 2023-01-23</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1r507:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>