<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:57:19 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-85275] Resharding oplog application should ignore DuplicateKey error</title>
                <link>https://jira.mongodb.org/browse/SERVER-85275</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Resharding like replication applies oplog entries in batches using multiple parallel threads. Oplog entries that touch the same document are batched together and applied in the same thread. So oplog application in resharding (and replication) preserves the (timestamp, _id) order; however, it doesn&apos;t preserve the overall write order. Consider a collection with a unique index {a: 1}, we insert the document {_id: 1, a: &quot;foo&quot;} and then delete {_id: 1, a: &quot;foo&quot;} and then insert {_id: 2, a: &quot;foo&quot;}. Resharding would apply the oplog entries in two threads:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Thread 1: insert {_id: 1, a: &quot;foo&quot;}, delete {_id: 1, a: &quot;foo&quot;}&lt;/li&gt;
	&lt;li&gt;Thread 2: insert {_id: 2 a: &quot;foo&quot;}&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;So if Thread 2 runs completely before Thread 1 if Thread 2 interleaves with Thread 1, then oplog application would end up with a DuplicateKey error. It should just ignore this DuplicateKey error just like what replication oplog application does today. &lt;/p&gt;</description>
                <environment></environment>
        <key id="2547024">SERVER-85275</key>
            <summary>Resharding oplog application should ignore DuplicateKey error</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="12452" iconUrl="https://jira.mongodb.org/images/icons/statuses/generic.png" description="">Needs Scheduling</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-server-cluster-scalability">Backlog - Cluster Scalability</assignee>
                                    <reporter username="cheahuychou.mao@mongodb.com">Cheahuychou Mao</reporter>
                        <labels>
                            <label>cs-product-sync</label>
                            <label>resharding-improvements</label>
                    </labels>
                <created>Tue, 16 Jan 2024 22:13:48 +0000</created>
                <updated>Thu, 1 Feb 2024 16:43:21 +0000</updated>
                                                                            <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="6059795" author="max.hirschhorn@10gen.com" created="Thu, 1 Feb 2024 16:43:21 +0000"  >&lt;p&gt;We decided to go forward with the mongosync approach. Resharding will create all secondary indexes as {unique: false} indexes and use the collMod procedure to convert relevant ones to {unique: true} indexes (&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-61158&quot; title=&quot;Convert a non-unique index to a unique index via the collMod command&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-61158&quot;&gt;&lt;del&gt;SERVER-61158&lt;/del&gt;&lt;/a&gt;) during resharding&apos;s critical section after writes on the recipient shard quiesced. The speed of this conversion process must be assessed to confirm it is compatible with the current resharding critical section window.&lt;/p&gt;</comment>
                            <comment id="6017115" author="max.hirschhorn@10gen.com" created="Wed, 17 Jan 2024 13:31:28 +0000"  >&lt;p&gt;Alternatively, we could have resharding&apos;s oplog application phase create all secondary indexes as {unique: false} indexes and use the collMod procedure to convert relevant ones to {unique: true} indexes (&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-61158&quot; title=&quot;Convert a non-unique index to a unique index via the collMod command&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-61158&quot;&gt;&lt;del&gt;SERVER-61158&lt;/del&gt;&lt;/a&gt;) during resharding&apos;s critical section after writes on the recipient shard quiesced. The collMod procedure would confirm the resulting index is actually unique. This is the technique used by mongosync.&lt;/p&gt;

&lt;p&gt;There is new risk with relaxing index constraints in that it becomes possible for resharding to not only propagate existing index inconsistencies of the source collection but also&#8212;through bugs specific to resharding&apos;s implementation&#8212;introduce new index inconsistencies. For comparison, chunk migration does not relax index constraints. I feel like engaging with the Product team can be helpful here because there may be cases such as draining a shard where we don&apos;t want manual intervention to be required yet in doing so we must also accept indexes are not guaranteed to be consistent on the recipient shards following the shard&apos;s removal.&lt;/p&gt;</comment>
                            <comment id="6017064" author="max.hirschhorn@10gen.com" created="Wed, 17 Jan 2024 13:06:28 +0000"  >&lt;p&gt;In addition to not having resharding&apos;s oplog application phase fail due to a DuplicateKey error, we should similarly relax all index constraints during resharding&apos;s oplog application phase. Relaxing index constraints can be achieved with &lt;tt&gt;OperationContext::setEnforceConstraints(false)&lt;/tt&gt; and is the technique used by replication&apos;s secondary oplog application as well as TenantOplogApplier for Serverless.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="26583"><![CDATA[Cluster Scalability]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 17 Jan 2024 13:06:28 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-cluster-scalability</customfieldvalue>
            <customfieldvalue>cheahuychou.mao@mongodb.com</customfieldvalue>
            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i38f7b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i2q4kc:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i381cn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>