<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:22:00 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-72502] Implement oplog batcher changes for parallelizing prepared transaction application</title>
                <link>https://jira.mongodb.org/browse/SERVER-72502</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;This should change &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d824f5a6e46ac74967e38e015bc46858c235a788/src/mongo/db/repl/oplog_batcher.cpp#L132&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;OplogBatcher::mustProcessIndividually&lt;/a&gt; so that prepare entries will no longer be required to be in their own batch. &lt;del&gt;The commitTransaction/abortTransaction command will no longer be required to be in its own batch, if and only if that commitTransaction/abortTransaction refers to a session in the SplitPrepareSessionManager.&lt;/del&gt;&lt;/p&gt;

&lt;p&gt;Note that currently &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d824f5a6e46ac74967e38e015bc46858c235a788/src/mongo/db/repl/oplog_batcher.cpp#L132&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;OplogBatcher::mustProcessIndividually&lt;/a&gt; is the same for both secondary mode and initial sync/recovering mode. We should probably do the above only for secondary mode and keep the initial sync/recovering behavior.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Note: We decided to still have every commitTransaction/abortTransaction entry to be processed in its own batch. This is because:&lt;/p&gt;

&lt;p&gt;1) Since we currently disallow a session to be split if it was already split and not yet released. This could be relaxed so that multiple splits on the same session are allowed as long as they refer to different txnNumbers, but before we do that, the following case can happen if we allow commits/aborts to be in the same batch with others:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;batch 1:&#160;&lt;tt&gt;&lt;span class=&quot;error&quot;&gt;&amp;#91;prepare(sess1, txn1)&amp;#93;&lt;/span&gt;&lt;/tt&gt;&lt;/li&gt;
	&lt;li&gt;batch 2:&#160;&lt;tt&gt;&lt;span class=&quot;error&quot;&gt;&amp;#91;commit(sess1, txn1), prepare(sess1, txn2)&amp;#93;&lt;/span&gt;&#160;&lt;/tt&gt;&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;When trying to split the second prepare, it fails due to this limitation since we only release the split sessions after the txn is committed and the batch is finished.&lt;/p&gt;

&lt;p&gt;2) Allowing commits/aborts to be in the same batch with other ops makes it possible for more edge cases and requires more testings. And since applying a commitTransaction/abortTransaction entry is usually quick, we think it&apos;s fine to first disallow this and once we&apos;re confident that the new algorithm works fine, we can start an effort to optimize it.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2223446">SERVER-72502</key>
            <summary>Implement oplog batcher changes for parallelizing prepared transaction application</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="wenbin.zhu@mongodb.com">Wenbin Zhu</assignee>
                                    <reporter username="wenbin.zhu@mongodb.com">Wenbin Zhu</reporter>
                        <labels>
                    </labels>
                <created>Wed, 4 Jan 2023 06:49:59 +0000</created>
                <updated>Sun, 29 Oct 2023 21:28:41 +0000</updated>
                            <resolved>Wed, 5 Apr 2023 15:55:32 +0000</resolved>
                                                    <fixVersion>7.0.0-rc0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="5325650" author="xgen-internal-githook" created="Wed, 5 Apr 2023 07:16:00 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;Wenbin Zhu&apos;, &apos;email&apos;: &apos;wenbin.zhu@mongodb.com&apos;, &apos;username&apos;: &apos;WenbinZhu&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-72502&quot; title=&quot;Implement oplog batcher changes for parallelizing prepared transaction application&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-72502&quot;&gt;&lt;del&gt;SERVER-72502&lt;/del&gt;&lt;/a&gt; Implement OplogBatcher changes for parallelizing prepared transaction oplog application.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/75228bcd11ef005a55816d76e6c336448a9e1931&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/75228bcd11ef005a55816d76e6c336448a9e1931&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5107632" author="JIRAUSER1258790" created="Wed, 11 Jan 2023 23:41:12 +0000"  >&lt;p&gt;We should be careful with the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d824f5a6e46ac74967e38e015bc46858c235a788/src/mongo/db/repl/oplog_batcher.cpp#L138&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;isEndOfLargeTransaction()&lt;/a&gt; check in mustProcessIndividually(). This check &lt;a href=&quot;https://github.com/mongodb/mongo/blob/0a856b4758c6a8ca12b19730a17c467da8777e87/src/mongo/db/repl/oplog_entry.cpp#L523-L539&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;evaluates to true&lt;/a&gt; for a prepareTransaction entry of&#160; a large prepared transaction, which would make every prepareTransaction entry of&#160; a large prepared transaction to be processed in its own batch. Our goal is to apply prepareTransaction entries concurrently with other entries, but it might be fine for prepareTransaction of&#160; a large prepared transaction to still be in its own batch, since we already parallelize application within a prepared transaction, and a large transaction means that we are applying more than 16MB worth of data, so it&apos;s fine to cut the batch.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="2223429">SERVER-72499</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="2205355">SERVER-71938</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25128"><![CDATA[Replication]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 3 Apr 2023 05:28:45 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        44 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-72499'>SERVER-72499</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-3093</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            44 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>wenbin.zhu@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1oy6f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i17hhs:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="6860">Repl 2023-01-23</customfieldvalue>
    <customfieldvalue id="6862">Repl 2023-02-20</customfieldvalue>
    <customfieldvalue id="7168">Repl 2023-04-03</customfieldvalue>
    <customfieldvalue id="7169">Repl 2023-04-17</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1okbr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>