<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:56:11 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-40870] OpObserverImpl::onTransactionPrepare() reserves OplogSlots rather than using the one reserved by its caller</title>
                <link>https://jira.mongodb.org/browse/SERVER-40870</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Large transactions will pack multiple operations into a single applyOps. The caller of&#160;OpObserver::onTransactionPrepare() won&apos;t know how many oplog slots are needed in advance. To avoid the bug in&#160;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-35798&quot; title=&quot;Writing an oplog entry for prepare should not push the lastApplied timestamp&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-35798&quot;&gt;&lt;del&gt;SERVER-35798&lt;/del&gt;&lt;/a&gt;, we still need to reserve an oplog slot in TransactionParticipant to&#160;prevent advancing&#160;lastApplied and prevent secondaries to read the prepare oplog entry before the transaction is actually prepared. OpObserverImpl can reserve and use its own OplogSlots to write packed oplog entries in side transactions. The prepare timestamp can be set by OpObserver or by TransactionParticipant with the&#160;getLastOp() on ReplClientInfo.&lt;/p&gt;

&lt;p&gt;An alternative solution is to reserve more oplog slots than needed, e.g. one for each operations, but the first proposal imposes less restrictions on the caller and simplifies the OpObserver interface.&lt;/p&gt;</description>
                <environment></environment>
        <key id="746410">SERVER-40870</key>
            <summary>OpObserverImpl::onTransactionPrepare() reserves OplogSlots rather than using the one reserved by its caller</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="siyuan.zhou@mongodb.com">Siyuan Zhou</assignee>
                                    <reporter username="siyuan.zhou@mongodb.com">Siyuan Zhou</reporter>
                        <labels>
                            <label>bigtxns_packing</label>
                    </labels>
                <created>Fri, 26 Apr 2019 23:07:44 +0000</created>
                <updated>Mon, 13 May 2019 18:21:59 +0000</updated>
                            <resolved>Thu, 9 May 2019 21:43:47 +0000</resolved>
                                                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="2244846" author="siyuan.zhou@10gen.com" created="Mon, 13 May 2019 18:21:59 +0000"  >&lt;p&gt;I believe we are on the same page. In &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=matthew.russotto&quot; class=&quot;user-hover&quot; rel=&quot;matthew.russotto&quot;&gt;matthew.russotto&lt;/a&gt;&apos;s &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-40797&quot; title=&quot;Write each applyOps in its own WUOW for prepare on primary&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-40797&quot;&gt;&lt;del&gt;SERVER-40797&lt;/del&gt;&lt;/a&gt;, we write &quot;partialTxn&quot; oplog entries before calling prepare as you mentioned, then reserve the oplog slot for the last applyOps oplog entry with &quot;prepare&quot; field, set prepareTimestamp(), call prepare() and write the last entry. Setting prepareTimestamp() and calling prepare() are currently done in transaction participant and will be moved to OpObserver. That is a layer violation but is the simplest solution we can think of. An alternative would be adding a new OpObserver function to write &quot;partialTxn&quot; entries, which is a mush bigger layer violation.&lt;/p&gt;</comment>
                            <comment id="2244137" author="judah.schvimer" created="Mon, 13 May 2019 12:36:41 +0000"  >&lt;blockquote&gt;
&lt;p&gt;it exposes a &quot;prepared&quot; state in the oplog which didn&apos;t exist on the primary&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;I agree, this would be a problem. This, however, only means we cannot write the final &quot;prepare&quot; oplog entry before calling prepare. We could still write &quot;partialTxn&quot; oplog entries if that were helpful.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;The &quot;partialTxn&quot; entries don&apos;t need to reserve oplog slots. Only the last one for implicit prepare should which determines the prepareTimestamp.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;How will we give the &quot;partialTxn&quot; entries optimes? Will they be written into the oplog before reserving the prepare oplog slot?&lt;/p&gt;</comment>
                            <comment id="2242860" author="siyuan.zhou@10gen.com" created="Fri, 10 May 2019 18:26:19 +0000"  >&lt;blockquote&gt;&lt;p&gt;Preparing a transaction will guarantee a subsequent commit will succeed. Only commit and rollback are allowed on a transaction after it has been prepared. The transaction prepare API is designed to support MongoDB exclusively, and guarantees update conflicts have been resolved, but does not guarantee durability.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;According to the WT document, that seems to imply&#160;update conflicts will be returned. When prepare() fails, I&apos;m wondering whether we can write the prepare entry. The contract of oplog is the corresponding operations should happen as if they are in the oplog order. I&apos;m afraid even if an abort follows the prepare (perhaps after some concurrent ops), it exposes a &quot;prepared&quot; state in the oplog which didn&apos;t exist on the primary. That seems a violation of the contract.&lt;/p&gt;

&lt;p&gt;The &quot;partialTxn&quot; entries don&apos;t need to reserve oplog slots. Only the last one for implicit prepare should which determines the prepareTimestamp.&lt;/p&gt;</comment>
                            <comment id="2241326" author="judah.schvimer" created="Fri, 10 May 2019 13:52:07 +0000"  >&lt;blockquote&gt;
&lt;p&gt;hit the same write conflict&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;While prepare can fail, I don&apos;t think it can get a write conflict.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;only reserve the oplog slot for the last applyOps&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;Can you clarify how this will work in more depth?&lt;/p&gt;</comment>
                            <comment id="2240674" author="siyuan.zhou@10gen.com" created="Thu, 9 May 2019 21:43:24 +0000"  >&lt;p&gt;Discussed with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=matthew.russotto&quot; class=&quot;user-hover&quot; rel=&quot;matthew.russotto&quot;&gt;matthew.russotto&lt;/a&gt;, we cannot write oplog entries before calling prepare() on the WUOW since prepare() can fail. Once it fails, we cannot write oplog entry no matter we write an abort immediately after that or not, otherwise, secondaries will pick up the prepare entry and hit the same write conflict.&lt;/p&gt;

&lt;p&gt;As part of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-40797&quot; title=&quot;Write each applyOps in its own WUOW for prepare on primary&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-40797&quot;&gt;&lt;del&gt;SERVER-40797&lt;/del&gt;&lt;/a&gt;, we will move the WUOW::prepare() into OpObserver and only reserve the oplog slot for the last applyOps, which will has a &quot;prepare&quot; field in applyOps format.&lt;/p&gt;

&lt;p&gt;Closing this as &quot;Won&apos;t Fix&quot;.&lt;/p&gt;</comment>
                            <comment id="2227464" author="judah.schvimer" created="Mon, 29 Apr 2019 13:21:24 +0000"  >&lt;p&gt;I want to mention a third solution &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=siyuan.zhou&quot; class=&quot;user-hover&quot; rel=&quot;siyuan.zhou&quot;&gt;siyuan.zhou&lt;/a&gt; and I discussed and rejected: Construct the applyOps entries we will eventually log, reserve their optimes, and then log them with the correct optimes. We then do not need to reserve extra oplog slots. This exposes an undesirable amount of &lt;tt&gt;OpObserver&lt;/tt&gt; behavior, however.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="744096">SERVER-40797</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 29 Apr 2019 13:21:24 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 39 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1035</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>siyuan.zhou@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 39 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>
            <customfieldvalue>siyuan.zhou@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huvuh3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hul8tj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2919">Repl 2019-05-06</customfieldvalue>
    <customfieldvalue id="2920">Repl 2019-05-20</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huvgqf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>