<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:20:53 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-72127] Upsert behaving differently on clustered collections compared with regular</title>
                <link>https://jira.mongodb.org/browse/SERVER-72127</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;When upsert fallback to insert it can cause a DuplicateKey error thrown to the user if there&apos;s another concurrent insert/upsert on the same key when the collection is clustered.&lt;/p&gt;

&lt;p&gt;Should upsert retry the update when encountering DuplicateKey?&lt;/p&gt;</description>
                <environment></environment>
        <key id="2211318">SERVER-72127</key>
            <summary>Upsert behaving differently on clustered collections compared with regular</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13202">Works as Designed</resolution>
                                        <assignee username="jordi.olivares-provencio@mongodb.com">Jordi Olivares Provencio</assignee>
                                    <reporter username="henrik.edin@mongodb.com">Henrik Edin</reporter>
                        <labels>
                    </labels>
                <created>Wed, 14 Dec 2022 20:36:46 +0000</created>
                <updated>Fri, 27 Oct 2023 13:51:48 +0000</updated>
                            <resolved>Wed, 22 Mar 2023 09:49:52 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="5291145" author="louis.williams" created="Wed, 22 Mar 2023 09:00:49 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jordi.olivares-provencio%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;jordi.olivares-provencio@mongodb.com&quot;&gt;jordi.olivares-provencio@mongodb.com&lt;/a&gt; thanks for the explanation here. I think it would be safe to close this as &quot;Works as Designed&quot;.&lt;/p&gt;</comment>
                            <comment id="5271779" author="JIRAUSER1264163" created="Tue, 14 Mar 2023 16:50:37 +0000"  >&lt;p&gt;A clustered collection inevitably returns a DuplicateKey error that &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a7b6d228fdc385fc6febb8cac8a43ec6ed96a8f3/src/mongo/db/catalog/collection_write_path.cpp#L193&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;emulates having a unique _id index&lt;/a&gt; if there is a duplicate record conflict. This means that the retry mechanism already is agnostic to the collection being clustered or not. In fact, there is no difference from the retry logic point of view between the two of them.&lt;/p&gt;

&lt;p&gt;Accordingly, the retry logic as described by &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-14322&quot; title=&quot;Retry on predicate unique index violations of update + upsert -&amp;gt; insert when possible&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-14322&quot;&gt;&lt;del&gt;SERVER-14322&lt;/del&gt;&lt;/a&gt; will retry this logic if there is reasonable belief that it is safe to do so. Otherwise we leave it to the application code to decide what to do with the error. This is crucial.&lt;/p&gt;

&lt;p&gt;If we were to treat the upsert as a logical transaction that fails with a WriteConflictException it would be equivalent to retrying the upsert by default as suggested by this ticket. This however contradicts with the previous ticket, as we are taking away the application&apos;s opportunity to decide what to do with this error. As such, it would potentially be a breaking change of current behaviour to existing users.&lt;/p&gt;

&lt;p&gt;For example, in the case of collections with only an _id unique index, an upsert with {{query: {_id: &amp;lt;key&amp;gt;, field1: &amp;lt;value1&amp;gt;} }} would go from returning a duplicate key error to potentially retrying it. Thus we could break an application&apos;s logic.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=henrik.edin%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;henrik.edin@mongodb.com&quot;&gt;henrik.edin@mongodb.com&lt;/a&gt; The scenario you describe here is already the case if the user provides an _id query, so the scenario you describe in the comment is already the case today. Can you clarify what scenario you found were the behaviour of a normal collection versus a clustered one is different?&lt;/p&gt;</comment>
                            <comment id="5059685" author="louis.williams" created="Fri, 16 Dec 2022 09:30:01 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=henrik.edin%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;henrik.edin@mongodb.com&quot;&gt;henrik.edin@mongodb.com&lt;/a&gt; you raise a good point. We actually already have code the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/11b5cafdb964e68eb892e413ee893bdb20a7574e/src/mongo/db/ops/write_ops_exec.cpp#L927&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;retries upserts on DuplicateKey errors&lt;/a&gt; for that reason. So we could look into modifying that code to handle clustered indexes.&lt;/p&gt;

&lt;p&gt;I had actually assumed that upsert performs the update an insert attempt in the same WUOW, but after looking at &lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/db/exec/upsert_stage.cpp&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;the code&lt;/a&gt;, we do not. If we did, then the case you described should not happen because the storage engine would just generate a write conflict.&lt;/p&gt;

&lt;p&gt;I actually think we could eliminate the need for any DuplicateKey retry handling (see &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-14322&quot; title=&quot;Retry on predicate unique index violations of update + upsert -&amp;gt; insert when possible&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-14322&quot;&gt;&lt;del&gt;SERVER-14322&lt;/del&gt;&lt;/a&gt;) for all collections (not just clustered) by performing the update and insert attempt in the same WUOW. &lt;/p&gt;</comment>
                            <comment id="5058133" author="henrik.edin" created="Thu, 15 Dec 2022 17:53:07 +0000"  >&lt;p&gt;Is that really correct &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=louis.williams%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;louis.williams@mongodb.com&quot;&gt;louis.williams@mongodb.com&lt;/a&gt;? If you are doing increments or something like that it doesn&apos;t seem correct to just overwrite and report both operations as succeeded. If we instead translate DuplicateKey in this case as WCE I believe the retry would do what is expected.&lt;/p&gt;</comment>
                            <comment id="5056768" author="louis.williams" created="Thu, 15 Dec 2022 09:23:37 +0000"  >&lt;p&gt;We should probably use &quot;overwrite=true&quot; when upserting records into clustered collections, rather than &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d3362e6dcef0d22074ef6c4aff8f6fb64816d238/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp#L1593-L1595&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;overwrite=false for all operations&lt;/a&gt;.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="143209">SERVER-14322</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25136"><![CDATA[Storage Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[5006R00001owf4oQAA]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 15 Dec 2022 09:23:37 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        46 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-2884</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            46 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>henrik.edin@mongodb.com</customfieldvalue>
            <customfieldvalue>jordi.olivares-provencio@mongodb.com</customfieldvalue>
            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1mvd3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i1j2hg:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="6594">Execution Team 2023-02-20</customfieldvalue>
    <customfieldvalue id="6685">Execution Team 2023-02-06</customfieldvalue>
    <customfieldvalue id="6773">Execution Team 2023-03-06</customfieldvalue>
    <customfieldvalue id="6816">Execution Team 2023-03-20</customfieldvalue>
    <customfieldvalue id="6819">Execution Team 2023-04-03</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1mhif:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>