<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:11:38 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-46487] The mongos routing for scatter/gather ops can have unbounded latency</title>
                <link>https://jira.mongodb.org/browse/SERVER-46487</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;div class=&quot;panel&quot; style=&quot;background-color: #eeeeee;border-color: #cccccc;border-width: 1px;&quot;&gt;&lt;div class=&quot;panelHeader&quot; style=&quot;border-bottom-width: 1px;border-bottom-color: #cccccc;background-color: #6cb33f;&quot;&gt;&lt;b&gt;Issue Status as of April 2020&lt;/b&gt;&lt;/div&gt;&lt;div class=&quot;panelContent&quot; style=&quot;background-color: #eeeeee;&quot;&gt;
&lt;p&gt;&lt;b&gt;ISSUE SUMMARY&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;The mongoS routing of scatter/gather operations iterates over the boundaries for the collection&#8217;s chunks sorted in ascending order. As the list is iterated, shards that own a chunk for the collection are discovered. If all shards in the cluster that own a chunk for the collection are discovered partway through the iteration, the iteration early-exits and the mongoS targets all of these shards.&lt;/p&gt;

&lt;p&gt;This logic is suboptimal if a large number of the sorted chunks are initially owned by a subset of shards. This can result in a large number of iterations needed to discover all shards. As an example, on a 2-shard cluster with a collection where the first 100k sorted chunks are owned on shard 1, the mongoS iterates over the boundaries for 100k chunks when routing a scatter/gather operation.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;USER IMPACT&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;On a sharded collection where a large number of the sorted chunks (e.g. 100k) are initially owned by a subset of shards, a scatter/gather operation can display increased latency (e.g. milliseconds) or increased mongoS CPU utilization. &lt;br/&gt;
 This chunk distribution can result from adding a shard to the cluster, as contiguous low chunk ranges get migrated into the new shard.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;WORKAROUND&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;Rearrange the chunks in a collection so that the &lt;em&gt;initial&lt;/em&gt; chunks in the sorted list &lt;em&gt;discover&lt;/em&gt; all the shards. The script below can be used to verify the number of iterations required to route a scatter/gather operation.&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;var ns = &apos;mydb.mycoll&apos;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;var nShards = db.getSiblingDB(&apos;config&apos;).shards.count();&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;var count = 0;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;var shards = [];&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;print(&apos;Iterating &apos; + ns + &apos; chunks sorted in ascending order...&apos;)&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;var cursor = db.getSiblingDB(&apos;config&apos;).chunks.aggregate([{$match : {ns: ns}}, {$sort : { min : 1 }}]);&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;while (cursor.hasNext() &amp;amp;&amp;amp; shards.length != nShards) {&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;   let nextShard = cursor.next().shard;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;   if (!shards.includes(nextShard)) {&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;      shards.push(nextShard);&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;      print(&quot;  discovered shard &quot; + nextShard + &quot; at chunk iteration &quot; + count)&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;   };&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;   ++count;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;print(&apos;Done. &apos; + count + &apos; chunk iterations to route a scatter/gather operation.&apos;)&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;/div&gt;&lt;/div&gt;</description>
                <environment></environment>
        <key id="1203695">SERVER-46487</key>
            <summary>The mongos routing for scatter/gather ops can have unbounded latency</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="gregory.noma@mongodb.com">Gregory Noma</assignee>
                                    <reporter username="josef.ahmad@mongodb.com">Josef Ahmad</reporter>
                        <labels>
                    </labels>
                <created>Fri, 28 Feb 2020 16:21:01 +0000</created>
                <updated>Sun, 29 Oct 2023 22:11:33 +0000</updated>
                            <resolved>Thu, 16 Apr 2020 22:11:40 +0000</resolved>
                                    <version>3.6.0</version>
                    <version>4.0.0</version>
                    <version>4.2.0</version>
                                    <fixVersion>4.0.19</fixVersion>
                    <fixVersion>4.2.7</fixVersion>
                    <fixVersion>3.6.19</fixVersion>
                    <fixVersion>4.4.0-rc3</fixVersion>
                    <fixVersion>4.7.0</fixVersion>
                                    <component>Sharding</component>
                                        <votes>4</votes>
                                    <watches>28</watches>
                                                                                                                <comments>
                            <comment id="3221351" author="xgen-internal-githook" created="Wed, 24 Jun 2020 13:22:06 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Gregory Noma&apos;, &apos;email&apos;: &apos;gregory.noma@gmail.com&apos;, &apos;username&apos;: &apos;gregorynoma&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-46487&quot; title=&quot;The mongos routing for scatter/gather ops can have unbounded latency&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-46487&quot;&gt;&lt;del&gt;SERVER-46487&lt;/del&gt;&lt;/a&gt; Add fast path to getShardIdsForRange() when chunk range is &lt;span class=&quot;error&quot;&gt;&amp;#91;MinKey, MaxKey&amp;#93;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;(cherry picked from commit d9d92a15561dc84223d302475e09d0fa91309550)&lt;br/&gt;
Branch: v3.6&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/1dc3198b115c4c0aaee87e2cec56b65c5c4ac620&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/1dc3198b115c4c0aaee87e2cec56b65c5c4ac620&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3053330" author="xgen-internal-githook" created="Thu, 23 Apr 2020 19:32:51 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Gregory Noma&apos;, &apos;email&apos;: &apos;gregory.noma@gmail.com&apos;, &apos;username&apos;: &apos;gregorynoma&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-46487&quot; title=&quot;The mongos routing for scatter/gather ops can have unbounded latency&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-46487&quot;&gt;&lt;del&gt;SERVER-46487&lt;/del&gt;&lt;/a&gt; Add fast path to getShardIdsForRange() when chunk range is &lt;span class=&quot;error&quot;&gt;&amp;#91;MinKey, MaxKey&amp;#93;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;(cherry picked from commit d9d92a15561dc84223d302475e09d0fa91309550)&lt;br/&gt;
Branch: v4.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/2124aa0b6673f4316a99af28c2eacc8c3d8b1441&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/2124aa0b6673f4316a99af28c2eacc8c3d8b1441&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3053172" author="xgen-internal-githook" created="Thu, 23 Apr 2020 18:49:09 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Gregory Noma&apos;, &apos;email&apos;: &apos;gregory.noma@gmail.com&apos;, &apos;username&apos;: &apos;gregorynoma&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-46487&quot; title=&quot;The mongos routing for scatter/gather ops can have unbounded latency&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-46487&quot;&gt;&lt;del&gt;SERVER-46487&lt;/del&gt;&lt;/a&gt; Add fast path to getShardIdsForRange() when chunk range is &lt;span class=&quot;error&quot;&gt;&amp;#91;MinKey, MaxKey&amp;#93;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;(cherry picked from commit d9d92a15561dc84223d302475e09d0fa91309550)&lt;br/&gt;
Branch: v4.4&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/ad478267c27b2b5f36cb39ad8c150081eaec9644&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/ad478267c27b2b5f36cb39ad8c150081eaec9644&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3053068" author="xgen-internal-githook" created="Thu, 23 Apr 2020 17:45:58 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Gregory Noma&apos;, &apos;email&apos;: &apos;gregory.noma@gmail.com&apos;, &apos;username&apos;: &apos;gregorynoma&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-46487&quot; title=&quot;The mongos routing for scatter/gather ops can have unbounded latency&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-46487&quot;&gt;&lt;del&gt;SERVER-46487&lt;/del&gt;&lt;/a&gt; Add fast path to getShardIdsForRange() when chunk range is &lt;span class=&quot;error&quot;&gt;&amp;#91;MinKey, MaxKey&amp;#93;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;(cherry picked from commit d9d92a15561dc84223d302475e09d0fa91309550)&lt;br/&gt;
Branch: v4.2&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/96d26475d7fb4ff6937c75b822b8ab573b63f860&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/96d26475d7fb4ff6937c75b822b8ab573b63f860&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3043472" author="xgen-internal-githook" created="Thu, 16 Apr 2020 21:53:48 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Gregory Noma&apos;, &apos;email&apos;: &apos;gregory.noma@gmail.com&apos;, &apos;username&apos;: &apos;gregorynoma&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-46487&quot; title=&quot;The mongos routing for scatter/gather ops can have unbounded latency&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-46487&quot;&gt;&lt;del&gt;SERVER-46487&lt;/del&gt;&lt;/a&gt; Add fast path to getShardIdsForRange() when chunk range is &lt;span class=&quot;error&quot;&gt;&amp;#91;MinKey, MaxKey&amp;#93;&lt;/span&gt;&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/d9d92a15561dc84223d302475e09d0fa91309550&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/d9d92a15561dc84223d302475e09d0fa91309550&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3038776" author="mingyan.zc@gmail.com" created="Tue, 14 Apr 2020 14:34:10 +0000"  >&lt;p&gt;Hi, Max,&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;Queries that aren&apos;t targeted by the shard key will be broadcasted to all shards which own chunks for the collection. The list of shards which own chunks for the collection corresponds to the keys of the&#160;&lt;tt&gt;RoutingTableHistory::_shardVersions&lt;/tt&gt;&#160;map. We&apos;re looking to introduce another optimization (again when the&#160;&lt;tt&gt;atClusterTime&lt;/tt&gt;&#160;read concern option hasn&apos;t been specified) such that if&#160;&lt;tt&gt;_rt-&amp;gt;overlappingRanges(min, max, true)&lt;/tt&gt;&#160;were going to cause all chunks to be iterated over, then we could instead populate the&#160;&lt;tt&gt;shardIds&lt;/tt&gt;&#160;set with the keys of the&#160;&lt;tt&gt;RoutingTableHistory::_shardVersions&lt;/tt&gt;&#160;map and skip doing the iteration itself.&#160;&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;This is exactly what I mean! I will upload my patch here for your reference. In fact, I didn&apos;t realize there is an existing&#160;checkAllElementsAreOfType(MinKey, ...) function so I made one in the BSONObj class again. Now that I can use this one instead!&lt;/p&gt;

&lt;p&gt;Thank you and please forward my regards to Esha!&lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/biggrin.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/p&gt;</comment>
                            <comment id="3038562" author="max.hirschhorn@10gen.com" created="Tue, 14 Apr 2020 13:01:30 +0000"  >&lt;p&gt;Assigning this one to you &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=gregory.noma&quot; class=&quot;user-hover&quot; rel=&quot;gregory.noma&quot;&gt;gregory.noma&lt;/a&gt;.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;This logic iterates over the chunks sorted by their string representation. As the sorted list is iterated, shards that own a chunk for the collection are discovered. If all shards in the cluster are discovered part-way through the iteration, the logic early exits and targets all shards.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;The early-exiting is referring to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/53b6694ef172411cfe8f5939b010efa35dd4d651/src/mongo/s/chunk_manager.cpp#L223-L230&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this part of &lt;tt&gt;ChunkManager::getShardIdsForRange()&lt;/tt&gt;&lt;/a&gt;. As noted in Cen Zhang&apos;s comment above, the &lt;tt&gt;!_clusterTime&lt;/tt&gt; part of that condition makes it so this optimization only applies when using the node&apos;s view of the latest routing information and not when the &lt;tt&gt;atClusterTime&lt;/tt&gt; read concern option has been specified.&lt;/p&gt;

&lt;p&gt;Queries that aren&apos;t targeted by the shard key will be broadcasted to all shards which own chunks for the collection. The list of shards which own chunks for the collection corresponds to the keys of the &lt;tt&gt;RoutingTableHistory::_shardVersions&lt;/tt&gt; map. We&apos;re looking to introduce another optimization (again when the &lt;tt&gt;atClusterTime&lt;/tt&gt; read concern option hasn&apos;t been specified) such that if &lt;tt&gt;_rt-&amp;gt;overlappingRanges(min, max, true)&lt;/tt&gt; were going to cause all chunks to be iterated over, then we could instead populate the &lt;tt&gt;shardIds&lt;/tt&gt; set with the keys of the &lt;tt&gt;RoutingTableHistory::_shardVersions&lt;/tt&gt; map and skip doing the iteration itself. You could think about it as being &lt;a href=&quot;https://github.com/mongodb/mongo/blob/53b6694ef172411cfe8f5939b010efa35dd4d651/src/mongo/s/chunk_manager.cpp#L275-L299&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;RoutingTableHistory::overlappingRanges(min, max, true)&lt;/tt&gt;&lt;/a&gt; returned {&lt;tt&gt;RoutingTableHistory::_chunkMap.begin(), RoutingTableHistory::_chunkMap.end()&lt;/tt&gt;}.&lt;/p&gt;

&lt;p&gt;I&apos;ll leave whether we want to express the optimization that way to you to figure out. Esha had also pointed me to the &lt;tt&gt;checkAllElementsAreOfType(MinKey, ...)&lt;/tt&gt; and &lt;tt&gt;checkAllElementsAreOfType(MaxKey, ...)&lt;/tt&gt; that we do &lt;a href=&quot;https://github.com/mongodb/mongo/blob/53b6694ef172411cfe8f5939b010efa35dd4d651/src/mongo/s/chunk_manager.cpp#L550-L551&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt; as a technique we could use to avoid calling &lt;tt&gt;RoutingTableHistory::overlappingRanges()&lt;/tt&gt; by comparing the &lt;tt&gt;min&lt;/tt&gt; and &lt;tt&gt;max&lt;/tt&gt; &lt;tt&gt;BSONObjs&lt;/tt&gt; directly. We&apos;ll likely want to backport these changes so their simplicity (e.g. ease of reasoning about their correctness) is also an important factor.&lt;/p&gt;

&lt;p&gt;As part of this work, we should add a microbenchmark to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/53b6694ef172411cfe8f5939b010efa35dd4d651/src/mongo/s/chunk_manager_refresh_bm.cpp&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;chunk_manager_refresh_bm.cpp&lt;/tt&gt;&lt;/a&gt; in order to demonstrate the performance improvement being made. I don&apos;t believe we have an existing case where &lt;tt&gt;ChunkManager::getShardIdsForRange()&lt;/tt&gt; is called for the range &lt;span class=&quot;error&quot;&gt;&amp;#91;MinKey, MaxKey&amp;#93;&lt;/span&gt;. The &lt;tt&gt;optimalShardSelector()&lt;/tt&gt; chunk distribution should see the most benefit because each shard owning a contiguous series of ranges in-order is when it&apos;d currently take the longest to see all shards by iterating over the ranges.&lt;/p&gt;</comment>
                            <comment id="3036264" author="mingyan.zc@gmail.com" created="Mon, 13 Apr 2020 06:44:54 +0000"  >&lt;p&gt;Hi,&#160;&lt;/p&gt;

&lt;p&gt;As I mentioned in&#160;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-47222&quot; title=&quot;Mongos high cpu usage on getShardIdsForRange while dealing shard key range query&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-47222&quot;&gt;&lt;del&gt;SERVER-47222&lt;/del&gt;&lt;/a&gt;,&#160;&lt;/p&gt;

&lt;p&gt;&quot;To resolve this issue, I think we can have a fast path for&#160;getShardIdsForRange() when we are not reading from a snapshot and the query range(all shard key fields) is&#160;&lt;span class=&quot;error&quot;&gt;&amp;#91;MinKey, MaxKey&amp;#93;&lt;/span&gt;, we only need to return all ShardIds through&#160;getAllShardIds().&quot;&lt;/p&gt;

&lt;p&gt;What do you think of this optimization? Is it possible? Looking forward for your feedback.&lt;/p&gt;

&lt;p&gt;Thanks!&lt;/p&gt;</comment>
                            <comment id="2921892" author="kvprasaddell@gmail.com" created="Mon, 2 Mar 2020 08:34:20 +0000"  >&lt;p&gt;Looking for a sooner resolution of this and hope to see it implemented in the upcoming patch.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="1299611">SERVER-47222</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="255953" name="mongos_routing_opt.patch" size="2535" author="mingyan.zc@gmail.com" created="Tue, 14 Apr 2020 14:36:31 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="18953"><![CDATA[v4.4]]></customfieldvalue>
    <customfieldvalue key="16775"><![CDATA[v4.2]]></customfieldvalue>
    <customfieldvalue key="15640"><![CDATA[v4.0]]></customfieldvalue>
    <customfieldvalue key="15141"><![CDATA[v3.6]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[5002K00000kpfsuQAA, 5002K00000mqZiaQAE, 5002K00000mrn7LQAQ, 5002K00000nmxlvQAA, 5002K00000uzOE1QAM]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 2 Mar 2020 08:34:20 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 33 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1771</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 33 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>mingyan.zc@gmail.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>gregory.noma@mongodb.com</customfieldvalue>
            <customfieldvalue>josef.ahmad@mongodb.com</customfieldvalue>
            <customfieldvalue>kvprasaddell@gmail.com</customfieldvalue>
            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hwxbb3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr5cnb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="3837">Sharding 2020-04-20</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hwwxkf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>