<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:07:10 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-44852] Have secondary oplog application handle transactions with commands</title>
                <link>https://jira.mongodb.org/browse/SERVER-44852</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Transactions appear as applyOps oplog entries. To allow transactions to include both commands, such as index and collection creation, and other operations, ensure that the entire transaction is applied as a single batch, and ensure that the operations inside this batch are applied serially. The latter goal can be accomplished by changing the logic in &lt;a href=&quot;https://github.com/mongodb/mongo/blob/ac99f0b697e689dc950a22def4c882c7b3bea9e4/src/mongo/db/repl/oplog_applier_impl.cpp#L838&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;OplogApplierImpl::fillWriterVectors&lt;/a&gt; to make sure that a single writer worker is used to apply the operations in the batch.&lt;/p&gt;

&lt;p&gt;To ensure the entire transaction is applied as a single batch, we can either parse the entire oplog entry as part of the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d1ba3bc8890f54e5dad91da85ce56626859c166c/src/mongo/db/repl/oplog_batcher.cpp#L130-L136&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;check&lt;/a&gt; for whether it must be processed individually to check for commands inside of transactions, or add information to the oplog entry to signify that it is a transaction with a command. The former may have be a slight performance hit due to the extra processing, but the latter is more difficult to implement. Provided the performance hit is not too high, the former solution is preferable.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1026484">SERVER-44852</key>
            <summary>Have secondary oplog application handle transactions with commands</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="maria.vankeulen@mongodb.com">Maria van Keulen</assignee>
                                    <reporter username="geert.bosch@mongodb.com">Geert Bosch</reporter>
                        <labels>
                    </labels>
                <created>Tue, 26 Nov 2019 20:29:30 +0000</created>
                <updated>Sun, 29 Oct 2023 22:14:34 +0000</updated>
                            <resolved>Tue, 14 Jan 2020 21:59:41 +0000</resolved>
                                                    <fixVersion>4.3.3</fixVersion>
                                    <component>Replication</component>
                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="2752226" author="maria.vankeulen" created="Wed, 22 Jan 2020 14:33:55 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=suganthi.mani&quot; class=&quot;user-hover&quot; rel=&quot;suganthi.mani&quot;&gt;suganthi.mani&lt;/a&gt; Yes, create_collection.js does &lt;a href=&quot;https://github.com/mongodb/mongo/blob/ef35c34b92e50e0438abeb2ffc83ff1472dd7d64/jstests/core/txns/create_collection.js#L17-L21&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;createCollection alongside CRUD ops&lt;/a&gt; in the same transaction. &lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-45370&quot; title=&quot;Test scenarios where multiple transactions try to create the same collection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-45370&quot;&gt;&lt;del&gt;SERVER-45370&lt;/del&gt;&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-44409&quot; title=&quot;Add FSM test for creating indexes/collections/databases in txns&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-44409&quot;&gt;&lt;del&gt;SERVER-44409&lt;/del&gt;&lt;/a&gt; will also add some more coverage of parallel transactions. Is there a specific test case you have in mind that isn&apos;t covered here? I&apos;m not sure I see the value in adding the same create_collection.js test to noPassthrough.&lt;/p&gt;</comment>
                            <comment id="2752032" author="judah.schvimer" created="Wed, 22 Jan 2020 14:04:38 +0000"  >&lt;p&gt;I think it is worth writing such a targeted test regardless so it is easier to debug if we break that case specifically, passthroughs are great for bridging coverage gaps, but they are also harder to debug.&lt;/p&gt;</comment>
                            <comment id="2748726" author="suganthi.mani" created="Tue, 21 Jan 2020 23:08:03 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=maria.vankeulen&quot; class=&quot;user-hover&quot; rel=&quot;maria.vankeulen&quot;&gt;maria.vankeulen&lt;/a&gt;  I do know we have jstests/core/txns/create_collection.js and some fsm workloads (like jstests/concurrency/fsm_workloads/create_database.js)  will run createCollection cmd inside the repl transaction.  Will those tests run the createCollection cmd along with the CRUD ops  inside a single transaction or run the createCollection cmd in one transaction and CRUD ops in other transaction? If not, I think we should try to write a jstest which should tests running a repl transaction containing  both createCollection cmd  and CRUD ops. Let me know your thoughts.&lt;/p&gt;

&lt;p&gt;CC &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=siyuan.zhou&quot; class=&quot;user-hover&quot; rel=&quot;siyuan.zhou&quot;&gt;siyuan.zhou&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2725393" author="xgen-internal-githook" created="Tue, 14 Jan 2020 21:33:03 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Maria van Keulen&apos;, &apos;email&apos;: &apos;maria.vankeulen@mongodb.com&apos;, &apos;username&apos;: &apos;mvankeulen94&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-44852&quot; title=&quot;Have secondary oplog application handle transactions with commands&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-44852&quot;&gt;&lt;del&gt;SERVER-44852&lt;/del&gt;&lt;/a&gt; Handle transactions with commands during oplog application&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/846c7aa84ac08dbccc7d727e9068406b7b2de033&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/846c7aa84ac08dbccc7d727e9068406b7b2de033&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2705117" author="maria.vankeulen" created="Mon, 6 Jan 2020 14:45:27 +0000"  >&lt;p&gt;I&apos;ve split out the work to accommodate cross-shard transactions into its own ticket (&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-45368&quot; title=&quot;Allow collection and index creation inside sharded_jscore_txns and sharded_causally_consistent_jscore_txns_passthrough suites&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-45368&quot;&gt;&lt;del&gt;SERVER-45368&lt;/del&gt;&lt;/a&gt;) since this present ticket has a fair amount of work scoped already.&lt;/p&gt;</comment>
                            <comment id="2657316" author="maria.vankeulen" created="Fri, 20 Dec 2019 14:52:03 +0000"  >&lt;p&gt;Thanks for letting me know. In that case, I agree that the best solution is to not decompose the applyOps for unprepared transactions if they contain commands. I will leave the serverStatus tracking as potential future work.&lt;/p&gt;</comment>
                            <comment id="2653675" author="judah.schvimer" created="Thu, 19 Dec 2019 22:34:46 +0000"  >&lt;blockquote&gt;
&lt;p&gt; they are all applied serially as part of one batch&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;This is only true for prepared transactions. Transactions that never go through prepare &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d1ba3bc8890f54e5dad91da85ce56626859c166c/src/mongo/db/repl/oplog_batcher.cpp#L130-L136&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;do not get applied in one batch&lt;/a&gt;. They get split onto writer threads as part of a larger &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d1ba3bc8890f54e5dad91da85ce56626859c166c/src/mongo/db/repl/oplog_applier_impl.cpp#L789-L832&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;batch&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="2653641" author="maria.vankeulen" created="Thu, 19 Dec 2019 22:11:32 +0000"  >&lt;p&gt;Per a discussion with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=suganthi.mani&quot; class=&quot;user-hover&quot; rel=&quot;suganthi.mani&quot;&gt;suganthi.mani&lt;/a&gt;, when we apply the operations inside a transaction, they are all applied serially as part of one batch. As such, applying both the commands and the operations inside the same batch should be fine. The work for this ticket should be to modify the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d1ba3bc8890f54e5dad91da85ce56626859c166c/src/mongo/db/repl/transaction_oplog_application.cpp#L60-L80&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;existing transaction oplog application code&lt;/a&gt; as necessary to accommodate oplog application of the commands newly allowed inside transactions.&lt;/p&gt;</comment>
                            <comment id="2570244" author="judah.schvimer" created="Wed, 27 Nov 2019 14:32:28 +0000"  >&lt;p&gt;I think it would be a lot simpler to process transaction oplog entries that contain commands in their own batch. It would be a perf hit, but I expect these are not very common so it might be quite small. We&apos;d have to either add an indicator to the oplog entry that it is a transaction that contains a command, or start looking at the entire transaction when batching oplog entries. I think it may be a good idea to start with a simpler solution and then add complexity if this is a significant bottleneck to secondary application in practice. Adding server status metrics counting how many transactions have commands in them could help us decide if this complexity is worth adding (and would be a useful metric in general).&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="1076550">SERVER-45368</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="974765">SERVER-44139</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 27 Nov 2019 14:32:28 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 3 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1168</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 3 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>geert.bosch@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>
            <customfieldvalue>maria.vankeulen@mongodb.com</customfieldvalue>
            <customfieldvalue>suganthi.mani@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hw6en3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hvuhlz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="3402">Execution Team 2019-12-30</customfieldvalue>
    <customfieldvalue id="3517">Execution Team 2020-01-13</customfieldvalue>
    <customfieldvalue id="3518">Execution Team 2020-01-27</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hw60wf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>