<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:28:43 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-52683] Relax restrictions for updates to new shard key fields during resharding</title>
                <link>https://jira.mongodb.org/browse/SERVER-52683</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;The changes from &lt;a href=&quot;https://github.com/mongodb/mongo/commit/6508f5d6ad5d514f433b660f0da7f513625973be&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;6508f5d&lt;/a&gt; as part of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-49825&quot; title=&quot;Replicate updates changing value under new shard key pattern as delete + insert in a transaction&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-49825&quot;&gt;&lt;del&gt;SERVER-49825&lt;/del&gt;&lt;/a&gt; reused the WouldChangeOwningShard error response handling in mongos. This comes with the following limitations:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;the update must be performed in a batch of size 1&lt;/li&gt;
	&lt;li&gt;the update must be performed as a retryable write or in a multi-document transaction&lt;/li&gt;
	&lt;li&gt;the update must not be performed as a multi=true update (this restriction was omitted from &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-49825&quot; title=&quot;Replicate updates changing value under new shard key pattern as delete + insert in a transaction&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-49825&quot;&gt;&lt;del&gt;SERVER-49825&lt;/del&gt;&lt;/a&gt; and maybe needs to be added)&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;One thought would be to have the donor shard handle the WouldChangeOwningShard exception locally without bubbling it up to mongos.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1536557">SERVER-52683</key>
            <summary>Relax restrictions for updates to new shard key fields during resharding</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="12300">Won&apos;t Do</resolution>
                                        <assignee username="backlog-server-sharding">[DO NOT USE] Backlog - Sharding Team</assignee>
                                    <reporter username="max.hirschhorn@mongodb.com">Max Hirschhorn</reporter>
                        <labels>
                            <label>PM-234-M2.5</label>
                            <label>PM-234-T-oplog-fetch</label>
                    </labels>
                <created>Sat, 7 Nov 2020 20:24:55 +0000</created>
                <updated>Tue, 6 Dec 2022 01:45:36 +0000</updated>
                            <resolved>Thu, 28 Jan 2021 01:33:23 +0000</resolved>
                                                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>1</watches>
                                                                                                                <comments>
                            <comment id="3586444" author="max.hirschhorn@10gen.com" created="Thu, 28 Jan 2021 01:33:23 +0000"  >&lt;p&gt;After discussing this ticket with Garaudy, we decided it isn&apos;t essential to relax these restrictions for updates to the shard key value under the new key pattern. These limitations will exist for updates on the shard key value after resharding completes. We will instead document these restrictions so users are made aware before they run reshardCollection.&lt;/p&gt;</comment>
                            <comment id="3517780" author="max.hirschhorn@10gen.com" created="Sun, 6 Dec 2020 14:47:44 +0000"  >&lt;blockquote&gt;
&lt;p&gt;One thought would be to have the donor shard handle the WouldChangeOwningShard exception locally without bubbling it up to mongos.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;One idea I recently had would be to&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/9dd51095f6ab6408af01656f7fa3f54da2661f4d/src/mongo/db/exec/update_stage.cpp#L572&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Continue to have UpdateStage detect when the current recipient shard isn&apos;t the same&lt;/a&gt; as the new recipient shard after the shard key value under the new pattern is modified.&lt;/li&gt;
	&lt;li&gt;But don&apos;t have UpdateStage &lt;a href=&quot;https://github.com/mongodb/mongo/blob/9dd51095f6ab6408af01656f7fa3f54da2661f4d/src/mongo/db/exec/update_stage.cpp#L596-L598&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;throw a WouldChangeOwningShard&lt;/a&gt; in this situation.&lt;/li&gt;
	&lt;li&gt;Instead UpdatedStage would stash the original document (or at least the shard key value under the current pattern and the destined recipient) on the CollectionUpdateArgs.&lt;/li&gt;
	&lt;li&gt;OpObserverImpl::onUpdate() would use the presence of the CollectionUpdateArgs member to generate an {&lt;tt&gt;applyOps: [{op: &quot;d&quot;}, {op: &quot;i&quot;}]&lt;/tt&gt;} oplog entry rather than &lt;a href=&quot;https://github.com/mongodb/mongo/blob/9dd51095f6ab6408af01656f7fa3f54da2661f4d/src/mongo/db/op_observer_impl.cpp#L544-L559&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;the op=&apos;u&apos; it would normally generate for the update&lt;/a&gt;. It should be safe for the primary to have performed the update differently than it&apos;ll give as instructions to secondaries so long as all of the writes are timestamped with the same value.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;I like this approach over trying to add local WouldChangeOwningShard exception handling to the donor shard&apos;s service entry point. I suspect it would be difficult to relax the batch of size 1 and multi=true restrictions in the WouldChangeOwningShard exception handling approach because the multiple updates may trigger the WouldChangeOwningShard exception multiple times for the same command.&lt;/p&gt;

&lt;p&gt;Another part of the OpObserverImpl::onUpdate() idea would be to avoid generating a new logical session ID for each multi-document transaction applyOps we&apos;d generate. I think it is possible to use a single logical session ID per donor shard. The challenge here is that OpObserverImpl::onUpdate() is called concurrently by updates to different documents within the same collection and cannot use the real-time thread scheduling order as an indication of the optime order that secondaries would end up seeing the oplog entries in. My idea here would be to use the timestamp component of the generated optime as the txnNumber. This guarantees the txnNumber appears as always increasing to secondaries. If we want to think slightly less hard about a single donor replica set shard members all using the same logical session ID (because the timestamp component of an optime is only guaranteed unique once it has become majority-committed), then we could make it so the donor shard primary generates a new logical session ID in each new term.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="1600208">SERVER-54040</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1601353">SERVER-54067</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1603083">SERVER-54096</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="1417724">SERVER-49825</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25141"><![CDATA[Sharding]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 12 Jan 2021 16:06:04 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 2 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-234</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 2 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-sharding</customfieldvalue>
            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hygg7b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hy2izj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4259">Sharding 2020-12-28</customfieldvalue>
    <customfieldvalue id="4482">Sharding 2021-01-11</customfieldvalue>
    <customfieldvalue id="4483">Sharding 2021-01-25</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10555" key="com.atlassian.jira.plugin.system.customfieldtypes:float">
                        <customfieldname>Story Points</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hyg2gn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>