<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:50:46 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-39017] Allow prepared transaction statements to persist in-memory until commit</title>
                <link>https://jira.mongodb.org/browse/SERVER-39017</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;h2&gt;&lt;a name=&quot;ProblemSummary&quot;&gt;&lt;/a&gt;Problem Summary&lt;/h2&gt;
&lt;p&gt;We would like to be able to observe all statements on transaction commit. This is so that migrations may take these statements on commit and add them to any current &lt;tt&gt;transferMods&lt;/tt&gt; queue.&lt;/p&gt;

&lt;p&gt;Currently, for prepared transactions, we discard statements &lt;a href=&quot;https://github.com/mongodb/mongo/blob/fcfe516250e2172d2fe6c9dab006b13d9d8140d7/src/mongo/db/op_observer_impl.cpp#L1016&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;on prepare&lt;/a&gt;, and don&apos;t persist these statements until commit time.&lt;/p&gt;

&lt;h2&gt;&lt;a name=&quot;ProposedApproach&quot;&gt;&lt;/a&gt;Proposed Approach&lt;/h2&gt;

&lt;p&gt;We will have to make changes to the transaction participant in order to persist these statements.&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;A new method &lt;tt&gt;retrieveOperations()&lt;/tt&gt; will simply retrieve and return a reference to &lt;tt&gt;_transactionOperations&lt;/tt&gt;. This method will require that the session is checked out, and that it is only called after the transaction has been prepared.&lt;/li&gt;
	&lt;li&gt;A new method &lt;tt&gt;endTransactionAndClearOperationsInMemory()&lt;/tt&gt; will clear the &lt;tt&gt;_transactionOperations&lt;/tt&gt; variable and set &lt;tt&gt;_transactionOperationBytes&lt;/tt&gt; to zero.&lt;/li&gt;
	&lt;li&gt;The current usage of the method &lt;tt&gt;endTransactionAndRetrieveOperations()&lt;/tt&gt; will be replaced with &lt;tt&gt;retrieveOperations()&lt;/tt&gt; when we want the operations, combined with &lt;tt&gt;endTransactionAndClearOperationsInMemory()&lt;/tt&gt; when we want to clear the transaction memory on commit.&lt;/li&gt;
	&lt;li&gt;Prepared transactions will no longer call &lt;tt&gt;endTransactionAndRetrieveOperations()&lt;/tt&gt;  on prepare. They will call both &lt;tt&gt;retrieveOperations()&lt;/tt&gt; and &lt;tt&gt;endTransactionAndClearOperationsInMemory()&lt;/tt&gt; on commit.&lt;/li&gt;
	&lt;li&gt;Non-prepared transactions will continue to persist the previous behavior of the previous method, calling both &lt;tt&gt;retrieveOperationsFor()&lt;/tt&gt; and &lt;tt&gt;endTransactionAndClearOperationsInMemory()&lt;/tt&gt; on commit.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;NOTE: We have renamed &lt;tt&gt;retrieveOperationsForMigrate()&lt;/tt&gt; to &lt;tt&gt;retrieveOperations()&lt;/tt&gt;, because we have identified usage of said operations outside of migrate, including &lt;a href=&quot;https://github.com/mongodb/mongo/blob/fcfe516250e2172d2fe6c9dab006b13d9d8140d7/src/mongo/db/op_observer_impl.cpp#L1037&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;
</description>
                <environment></environment>
        <key id="672882">SERVER-39017</key>
            <summary>Allow prepared transaction statements to persist in-memory until commit</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="blake.oler@mongodb.com">Blake Oler</assignee>
                                    <reporter username="blake.oler@mongodb.com">Blake Oler</reporter>
                        <labels>
                    </labels>
                <created>Tue, 15 Jan 2019 19:52:24 +0000</created>
                <updated>Sun, 29 Oct 2023 22:25:06 +0000</updated>
                            <resolved>Fri, 1 Feb 2019 19:27:05 +0000</resolved>
                                    <version>4.1.6</version>
                                    <fixVersion>4.1.8</fixVersion>
                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="2135318" author="xgen-internal-githook" created="Fri, 1 Feb 2019 19:26:22 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Blake Oler&apos;, &apos;email&apos;: &apos;blake.oler@mongodb.com&apos;, &apos;username&apos;: &apos;BlakeIsBlake&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-39017&quot; title=&quot;Allow prepared transaction statements to persist in-memory until commit&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-39017&quot;&gt;&lt;del&gt;SERVER-39017&lt;/del&gt;&lt;/a&gt; Allow prepared transaction statements to persist in-memory until commit&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/b8bfe9ff391ebeb10a5c2fb86979d854d17d0fd5&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/b8bfe9ff391ebeb10a5c2fb86979d854d17d0fd5&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2118682" author="blake.oler" created="Thu, 17 Jan 2019 19:08:26 +0000"  >&lt;p&gt;A note: the transactions team decided to call the &lt;tt&gt;onCommit&lt;/tt&gt; opObserver while an unprepared transaction would still be &lt;tt&gt;inProgress&lt;/tt&gt;. This is so that any exceptions inside the opObserver would allow the transaction to abort instead of committing. As such, we will maintain the current invariant of either &lt;tt&gt;inProgress&lt;/tt&gt; or &lt;tt&gt;Prepared&lt;/tt&gt; for both functions. This will result in no change to current behavior for the prerequisite states.&lt;/p&gt;</comment>
                            <comment id="2117375" author="judah.schvimer" created="Wed, 16 Jan 2019 19:18:27 +0000"  >&lt;p&gt;lgtm, I defer to Kal on the lifecycle management.&lt;/p&gt;</comment>
                            <comment id="2116286" author="kaloian.manassiev" created="Tue, 15 Jan 2019 21:14:07 +0000"  >&lt;p&gt;Yeah, I think &lt;tt&gt;retrieveCompletedTransactionOperations&lt;/tt&gt; might be a more appropriate name.&lt;/p&gt;

&lt;p&gt;I would make &lt;tt&gt;retrieveOperations()&lt;/tt&gt; invariant that the transaction&apos;s status is &lt;tt&gt;InPrepare&lt;/tt&gt; or &lt;tt&gt;Committed&lt;/tt&gt; so that callers are not allowed to obtain an intermediate view of the transaction.&lt;/p&gt;</comment>
                            <comment id="2116246" author="blake.oler" created="Tue, 15 Jan 2019 20:52:15 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev&quot;&gt;kaloian.manassiev&lt;/a&gt;&lt;/p&gt;

&lt;ol&gt;
	&lt;li&gt;The method &lt;tt&gt;retrieveOperations()&lt;/tt&gt; will have the same requirements as &lt;tt&gt;endTransactionAndRetrieveOperations()&lt;/tt&gt;. That is &amp;#8211; a session must be checked out. We can see this requirement at the &lt;tt&gt;onTransactionCommit()&lt;/tt&gt; handler &lt;a href=&quot;https://github.com/mongodb/mongo/blob/fcfe516250e2172d2fe6c9dab006b13d9d8140d7/src/mongo/db/op_observer_impl.cpp#L978&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;. And as a result of the prepared transaction lifecycle, a commit &lt;em&gt;must&lt;/em&gt; be after the participant has entered the prepared state, implying that the list of operations will have become static.&lt;/li&gt;
	&lt;li&gt;As far as I am aware, we don&apos;t &lt;em&gt;need&lt;/em&gt; to return a copy. A reference should just work. I will defer to using a reference unless something indicating otherwise comes up during implementation of this ticket.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;The above information will be integrated into the ticket description.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev&quot;&gt;kaloian.manassiev&lt;/a&gt; do we want to rename the method &lt;tt&gt;retrieveOperations()&lt;/tt&gt; to indicate that the operations are now static? Maybe &lt;tt&gt;retrieveCompletedTransactionOperations()&lt;/tt&gt;.&lt;/p&gt;</comment>
                            <comment id="2116176" author="kaloian.manassiev" created="Tue, 15 Jan 2019 20:09:29 +0000"  >&lt;p&gt;I have a couple of questions:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;A new method retrieveOperations() will simply retrieve and return a copy of _transactionOperations.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;What are the lifetime requirements for this method? I.e., it is allowed to only be called with the session checked-out, and only after the transaction participant enters the prepared state, right?&lt;/p&gt;

&lt;p&gt;With the lifetime above - does it need to return a copy or just reference would work? I think you can just return a reference. The reason why this is better is that this object can be up to 16MB in size.&lt;/p&gt;</comment>
                            <comment id="2116153" author="blake.oler" created="Tue, 15 Jan 2019 19:53:49 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt; can I get your LGTM on this approach? It has integrated your comments on the ticket &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-38284&quot; title=&quot;Remove donor collection X-lock acquisition for starting the clone phase&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-38284&quot;&gt;&lt;del&gt;SERVER-38284&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="639246">SERVER-38284</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="672920">SERVER-39021</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="2130157">SERVER-69478</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="2132836">SERVER-69560</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 15 Jan 2019 20:09:29 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 1 week, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1286</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 1 week, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>blake.oler@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huje8v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hu24ef:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2726">Sharding 2019-01-28</customfieldvalue>
    <customfieldvalue id="2786">Sharding 2019-02-11</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huj0i7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>