<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:36:04 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-14813] Upsert and Shard is tightly coupled, and there is no clear documentation on that</title>
                <link>https://jira.mongodb.org/browse/SERVER-14813</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;To perform &quot;upsert&quot; operation on a sharded MongoDB cluster, the query MUST exactly match the shard key. &lt;/p&gt;

&lt;p&gt;For example:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;update( {a:1, b:2}, { $setOnInsert { c : 3, d : 4}}, { upsert: true})&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;The above update does not work if the shard is key is for example &lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;{a: 1, _id: 1}. &lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Is this the designed behavior? If so, why cannot I not find the detailed example or warning for this? Since the shard kay cannot be modified, this is causing huge problems. It just completely broke our migration from a stand-alone server to the sharded cluster.&lt;/p&gt;</description>
                <environment></environment>
        <key id="151199">SERVER-14813</key>
            <summary>Upsert and Shard is tightly coupled, and there is no clear documentation on that</summary>
                <type id="6" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14720&amp;avatarType=issuetype">Question</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="thomas.rueckstiess@mongodb.com">Thomas Rueckstiess</assignee>
                                    <reporter username="lizhenyu2000">Zhenyu Li</reporter>
                        <labels>
                    </labels>
                <created>Thu, 7 Aug 2014 00:28:32 +0000</created>
                <updated>Mon, 25 Aug 2014 23:22:14 +0000</updated>
                            <resolved>Mon, 25 Aug 2014 23:22:14 +0000</resolved>
                                                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="703351" author="thomasr" created="Mon, 25 Aug 2014 23:22:02 +0000"  >&lt;p&gt;Hi Zhenyu,&lt;/p&gt;

&lt;p&gt;I&apos;m glad you were able to resolve the issue.&lt;/p&gt;

&lt;p&gt;I&apos;ve linked this ticket to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-4000&quot; title=&quot;command to change shard key of a collection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-4000&quot;&gt;&lt;del&gt;SERVER-4000&lt;/del&gt;&lt;/a&gt;, which is a request to have a command to change the shard key. Feel free to watch this ticket and vote for it to increase the priority.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Thomas&lt;/p&gt;</comment>
                            <comment id="692694" author="lizhenyu2000" created="Sat, 16 Aug 2014 21:54:12 +0000"  >&lt;p&gt;Thomas,&lt;/p&gt;

&lt;p&gt;Thanks a lot for your detailed reply.  The reason I was trying to use _id in my shard key is that I don&apos;t want one chunk to grow to large, and I want Mongo to split that chunk further. &lt;br/&gt;
The reason that I cannot use _id in my query is that all my documents have a randomly generated _id field. So, I cannot just get the _id field from the information I have on that particular document.&lt;/p&gt;

&lt;p&gt;I ended up doing the following to fix this issue:&lt;/p&gt;

&lt;p&gt;Dump all the data (more than 200GB) to the disk, and restore it to a new database, and re-shard it without using the _id key. After that, I was able to perform upsert without problems.&lt;/p&gt;

&lt;p&gt;If you can, it would be great to have the following:&lt;/p&gt;

&lt;p&gt;Have a tool to change the shard key somehow. I know it requires re-distributing all the data, but it would be very valuable to have this tool in the future.&lt;br/&gt;
I understand that you guys have the documentation here: &lt;a href=&quot;http://docs.mongodb.org/manual/reference/method/db.collection.update/#sharded-collections&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://docs.mongodb.org/manual/reference/method/db.collection.update/#sharded-collections&lt;/a&gt;&lt;br/&gt;
I just wish that there could be a warning and a link to the above reference on this page: &lt;a href=&quot;http://docs.mongodb.org/manual/tutorial/deploy-shard-cluster/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://docs.mongodb.org/manual/tutorial/deploy-shard-cluster/&lt;/a&gt;&lt;br/&gt;
After all, most of the people don&apos;t dig into the reference section as often as the tutorial section. &lt;/p&gt;

&lt;p&gt;Thanks,&lt;/p&gt;

&lt;p&gt;Zhenyu&lt;/p&gt;</comment>
                            <comment id="683837" author="thomasr" created="Thu, 7 Aug 2014 14:48:24 +0000"  >&lt;p&gt;Hi Zhenyu,&lt;/p&gt;

&lt;p&gt;The behavior you&apos;re seeing is expected. An update that does not specify {multi:true} has to contain the full shard key or the _id field in order to be routed to the correct shard. This is documented on our page about updates under the &lt;a href=&quot;http://docs.mongodb.org/manual/reference/method/db.collection.update/#sharded-collections&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Sharded Collections&lt;/a&gt; section.&lt;/p&gt;

&lt;p&gt;However, in the case of an &lt;em&gt;upsert&lt;/em&gt;, specifying {multi:true} in a sharded collection may still not work, as it would create documents on multiple shards that may end up being orphaned (i.e. not belonging to that shard). In my tests, an update in a sharded collection with {multi:true, upsert:true}, but without fully specifying the shard key in the query part, was still disallowed. &lt;/p&gt;

&lt;p&gt;Perhaps you can explain what you&apos;d like to achieve with the upsert without specifying the shard key, and what you&apos;d expect to happen if the document can&apos;t be found on some of the shards. Understanding the use case better, I may be able to suggest an alternative, or at the least improve our documentation around upsert behavior in sharded collections.&lt;/p&gt;

&lt;p&gt;One comment about this statement here:&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;the query MUST exactly match the shard key.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;This is almost correct. The shard key must be fully specified in the query part of the upsert. For shard key {a:1, _id:1}, you could for example run the following upsert:&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;update( {a:1, b:2, _id: 3}, { $setOnInsert: { c: 3, d: 4}}, { upsert: true })&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;


&lt;p&gt;Regards,&lt;br/&gt;
Thomas&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="23132">SERVER-4000</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 7 Aug 2014 14:48:24 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        9 years, 25 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>false</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            9 years, 25 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>thomas.rueckstiess@mongodb.com</customfieldvalue>
            <customfieldvalue>lizhenyu2000</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlqcv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hs15hj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>131223</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsawn3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>