<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:52:36 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-39624] Reordering of transaction requests from mongos can lead shards to unexpectedly have in-progress transactions</title>
                <link>https://jira.mongodb.org/browse/SERVER-39624</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;While running a sharded transaction, the router tracks in-memory each shard that has been involved in the transaction. When a new shard is targeted by a statement, the router adds it to the participant list as a pending participant and attaches &lt;tt&gt;startTransaction=true&lt;/tt&gt; and the active transaction&apos;s txnId to the next request sent to that shard.&lt;/p&gt;

&lt;p&gt;The shard is considered &quot;pending&quot; because of the shard version version protocol, which means the router won&apos;t know the shard was able to satisfy the request sent to it until it gets an OK response. To handle the case where a pending participant returns a stale version error, the router will abort the active txnId on each pending participant (not just the one that returned the error), wait for each to respond, then retry the current statement, possibly targeting a different set of shards. This allows the router to handle these errors within a transaction, instead of returning a transient error and making the client retry with a new txnId.&lt;/p&gt;

&lt;p&gt;To enable this behavior, shards can accept a request with &lt;tt&gt;startTransaction=true&lt;/tt&gt; more than once for a txnId, but only if the shard&apos;s local transaction is in the aborted state. This relies on the abort sent to every pending participant before retrying reaching each one after the first requests with &lt;tt&gt;startTransaction=true&lt;/tt&gt;, which is not guaranteed in an asynchronous network. If the abort arrives &lt;b&gt;before&lt;/b&gt; the first request on a shard and that shard is targeted by the retry, the retry will be rejected by that shard because it will have an in-progress transaction at that txnId.&lt;/p&gt;</description>
                <environment></environment>
        <key id="698905">SERVER-39624</key>
            <summary>Reordering of transaction requests from mongos can lead shards to unexpectedly have in-progress transactions</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="jack.mulrow@mongodb.com">Jack Mulrow</assignee>
                                    <reporter username="jack.mulrow@mongodb.com">Jack Mulrow</reporter>
                        <labels>
                            <label>ShardedTxn:RouterSupport</label>
                    </labels>
                <created>Fri, 15 Feb 2019 21:29:54 +0000</created>
                <updated>Sun, 29 Oct 2023 22:23:56 +0000</updated>
                            <resolved>Tue, 26 Feb 2019 15:44:05 +0000</resolved>
                                                    <fixVersion>4.1.9</fixVersion>
                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="2163549" author="xgen-internal-githook" created="Tue, 26 Feb 2019 21:07:28 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Jack Mulrow&apos;, &apos;email&apos;: &apos;jack.mulrow@mongodb.com&apos;, &apos;username&apos;: &apos;jsmulrow&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-39624&quot; title=&quot;Reordering of transaction requests from mongos can lead shards to unexpectedly have in-progress transactions&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-39624&quot;&gt;&lt;del&gt;SERVER-39624&lt;/del&gt;&lt;/a&gt; Allow finds to fail with StaleChunkHistory in snapshot_read_catalog_operations.js&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/76bea9daaf67deee934c50e6ebcb4e5d5bbb397e&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/76bea9daaf67deee934c50e6ebcb4e5d5bbb397e&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2163214" author="jack.mulrow" created="Tue, 26 Feb 2019 17:16:07 +0000"  >&lt;p&gt;A note from the review on why the router can still retry on the view resolution error even when the fail point isn&apos;t set:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;I didn&apos;t disable retries on the ViewResolution error because otherwise it would be impossible to read from a view in a transaction. I think this is fine though, because requests that can return this error must have been sent to a single shard (the primary for the view&apos;s database), so the router does not need to send abort before retrying on the view resolution error from a pending participant, which avoids the problem in the description.&lt;/p&gt;&lt;/blockquote&gt;</comment>
                            <comment id="2163043" author="xgen-internal-githook" created="Tue, 26 Feb 2019 15:43:28 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Jack Mulrow&apos;, &apos;email&apos;: &apos;jack.mulrow@mongodb.com&apos;, &apos;username&apos;: &apos;jsmulrow&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-39624&quot; title=&quot;Reordering of transaction requests from mongos can lead shards to unexpectedly have in-progress transactions&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-39624&quot;&gt;&lt;del&gt;SERVER-39624&lt;/del&gt;&lt;/a&gt; Put internal router retries for stale version and snapshot errors behind a fail point&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/b0d3c0d2934a2096ba27362db22fe768b5341485&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/b0d3c0d2934a2096ba27362db22fe768b5341485&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2157847" author="jack.mulrow" created="Wed, 20 Feb 2019 23:24:26 +0000"  >&lt;p&gt;As discussed at the sharded transactions standup, we&apos;ll fix this by preventing mongos from retrying on stale shard/db version and snapshot errors within a transaction at all and having it return a TransientTransactionError label instead, so the client will retry the whole transaction at a higher txnNumber. This should allow sharded transactions to work in all cases without much work. My proposed approach above will instead be considered as part of the Optimize Cross Shard Transactions epic in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-39704&quot; title=&quot;Allow mongos to retry on stale version and snapshot errors within a transaction&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-39704&quot;&gt;SERVER-39704&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="2154018" author="jack.mulrow" created="Fri, 15 Feb 2019 22:58:31 +0000"  >&lt;p&gt;Since the fundamental problem is that mongos can send multiple requests to the same shard with &lt;tt&gt;startTransaction=true&lt;/tt&gt; for a txnId, we might be able to fix this by adding another field to the txnId that is generated by mongos and transparent to the user. Then each time mongos sends a shard &lt;tt&gt;startTransaction=true&lt;/tt&gt;, it can generate a new value for this field (transaction version?) that shards can use to distinguish the earlier attempts. This field could be scoped to a particular txnNumber, so the existing machinery for client retries on transient transaction errors would still work, i.e. any comparison of txnIds always compares the txnNumber before this field.&lt;/p&gt;

&lt;p&gt;If we include each shard&apos;s expected version in the participant list and send it with prepare/commit (or every request within a transaction), we can abort the entire transaction if any shard has an unexpected version and return a transient transaction error label so the client can retry with a higher txnNumber, since this should only happen because of reordered messages if the client operates correctly.&lt;/p&gt;

&lt;p&gt;We could even get rid of the aborts between statement retries by making shards treat a higher transaction version the same as a higher txnNumber and overwrite any state from a previous attempt.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10520">
                    <name>Problem/Incident</name>
                                            <outwardlinks description="causes">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="701590">SERVER-39704</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 26 Feb 2019 15:43:28 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 50 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-834</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 50 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>20.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>jack.mulrow@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hunr27:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hubhl3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2824">Sharding 2019-03-11</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hundbj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>