<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 08:24:49 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[DRIVERS-2139] Test retryable writes against real shutdown scenarios</title>
                <link>https://jira.mongodb.org/browse/DRIVERS-2139</link>
                <project id="10980" key="DRIVERS">Drivers</project>
                    <description>&lt;p&gt;In&#160;&lt;a href=&quot;https://jira.mongodb.org/browse/PYTHON-1650&quot; title=&quot;Replica failover with retryWrites not always working&quot; class=&quot;issue-link&quot; data-issue-key=&quot;PYTHON-1650&quot;&gt;&lt;del&gt;PYTHON-1650&lt;/del&gt;&lt;/a&gt;, a user discovered a bug in PyMongo&apos;s implementation of retryable writes where it was possible for both the initial retryable write attempt and the retry attempt to never increment the txnNumber. Worse, this could occur when a primary was shut down normally.&#160;Testing retryable writes against real shut down scenarios (shutdowns, crashes, stepdowns) would have caught this issue.&lt;/p&gt;

&lt;p&gt;The specific scenario where PyMongo&apos;s bug would be hit is when:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;Primary shuts down.&lt;/li&gt;
	&lt;li&gt;Client initiates a retryable write, retryWrites=true, and the server is 3.6 or 4.0.&lt;/li&gt;
	&lt;li&gt;Server selection succeeds on the stale primary yielding server P.&lt;/li&gt;
	&lt;li&gt;Socket is checked out of P&apos;s pool.&lt;/li&gt;
	&lt;li&gt;The pool notices the socket has been closed (&lt;a href=&quot;https://github.com/mongodb/mongo-python-driver/blob/3.7.2/pymongo/pool.py#L1090-L1095&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;because this socket has been idle for &amp;gt;1 second&lt;/a&gt;) and attempts to create a new connection to the old primary.&lt;/li&gt;
	&lt;li&gt;The connection attempt fails with a (retryable) &quot;connection refused&quot; error because the old primary is not running.&lt;/li&gt;
	&lt;li&gt;The initial write attempt fails with a retryable error before incrementing the txnNumber.&lt;/li&gt;
	&lt;li&gt;The retry attempt waits until the new primary is elected, does not increment the txnNumber, and runs the write with a previous txnNumber.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;The fix implemented in &lt;a href=&quot;https://pypi.org/project/pymongo/3.7.2&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;PyMongo 3.7.2&lt;/a&gt; is to always increment the txnNumber &lt;b&gt;before&lt;/b&gt; server/socket selection (similar to how start_transaction works).&lt;/p&gt;

&lt;p&gt;Note that it would be much less likely for this to occur if &lt;a href=&quot;https://jira.mongodb.org/browse/PYTHON-1435&quot; title=&quot;Retry server selection if, after selection, a socket is determined to be unsuitable before a message is sent on it&quot; class=&quot;issue-link&quot; data-issue-key=&quot;PYTHON-1435&quot;&gt;PYTHON-1435&lt;/a&gt; was implemented because server selection would wait for the new primary on the initial attempt.&lt;/p&gt;</description>
                <environment></environment>
        <key id="617749">DRIVERS-2139</key>
            <summary>Test retryable writes against real shutdown scenarios</summary>
                <type id="14901" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14700&amp;avatarType=issuetype">Spec Change</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="shane.harvey@mongodb.com">Shane Harvey</reporter>
                        <labels>
                    </labels>
                <created>Fri, 12 Oct 2018 21:07:07 +0000</created>
                <updated>Thu, 21 Dec 2023 20:14:02 +0000</updated>
                                                                <component>Retryability</component>
                                        <votes>0</votes>
                                    <watches>8</watches>
                                                                                                                <comments>
                            <comment id="2858904" author="esha.bhargava" created="Mon, 10 Feb 2020 19:43:33 +0000"  >&lt;p&gt;This work will be done post 4.4&lt;/p&gt;</comment>
                            <comment id="2516639" author="prashant.mital" created="Mon, 4 Nov 2019 18:33:31 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jeff.yemin&quot; class=&quot;user-hover&quot; rel=&quot;jeff.yemin&quot;&gt;jeff.yemin&lt;/a&gt; Indeed it is. Thanks for pointing it out!&lt;/p&gt;</comment>
                            <comment id="2514926" author="jeff.yemin" created="Fri, 1 Nov 2019 21:04:50 +0000"  >&lt;p&gt;CC &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=prashant.mital&quot; class=&quot;user-hover&quot; rel=&quot;prashant.mital&quot;&gt;prashant.mital&lt;/a&gt; maybe relevant for planned maintenance testing scenarios.&lt;/p&gt;</comment>
                            <comment id="2032075" author="behackett" created="Fri, 12 Oct 2018 22:35:23 +0000"  >&lt;p&gt;That is precisely the bug, and what happened in the reproduction. Even worse, the initial write was an insert, the retry write was an update, and the server converted the reply to the retry (which was the reply from the successful insert) into an upsert...&lt;/p&gt;</comment>
                            <comment id="2032058" author="jesse" created="Fri, 12 Oct 2018 22:16:43 +0000"  >&lt;p&gt;So PyMongo could use a completely unrelated txnNumber for an operation?&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;collection.update_one(filter, update)  # Uses txnNumber 1.&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;# In between, the primary dies.&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;collection.insert_one(doc)  # Also uses txnNumber 1 after finding new primary.&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;</comment>
                            <comment id="2031998" author="shane.harvey" created="Fri, 12 Oct 2018 21:11:13 +0000"  >&lt;p&gt;We do have a single prose test that tests with a real replica set stepdown (&lt;a href=&quot;https://jira.mongodb.org/browse/DRIVERS-579&quot; title=&quot;Replica set failover test for retryable writes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DRIVERS-579&quot;&gt;&lt;del&gt;DRIVERS-579&lt;/del&gt;&lt;/a&gt;). It&apos;s possible this test would have caught this bug so I&apos;ll report back once we implement it in &lt;a href=&quot;https://jira.mongodb.org/browse/PYTHON-1657&quot; title=&quot;Replica set failover test for retryable writes	&quot; class=&quot;issue-link&quot; data-issue-key=&quot;PYTHON-1657&quot;&gt;&lt;del&gt;PYTHON-1657&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                                        </outwardlinks>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="614413">PYTHON-1650</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="688830">DRIVERS-2121</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                            <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10951" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Driver Changes</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10748"><![CDATA[Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hu0gjj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        </customfields>
    </item>
</channel>
</rss>