<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:11:26 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-68672] Query using shard key and document ID does not use available index</title>
                <link>https://jira.mongodb.org/browse/SERVER-68672</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I have a sharded collection with around ~ 200M documents, and an index on this collection: {datasetId: 1, _id: 1}, where datasetId is the shard key on the collection.&lt;/p&gt;

&lt;p&gt;When I query on the collection with the following query:&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;find({datasetId: &lt;/span&gt;&lt;span style=&quot;color: blue; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;&apos;YYY&apos;&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;, _id: ${gt: ObjectId(&lt;/span&gt;&lt;span style=&quot;color: blue; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;&apos;XXX&apos;&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;)}}.sort({_id: &lt;/span&gt;&lt;span style=&quot;color: #009900; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;1&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;})&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;The query does not use the aforementioned index, it uses the {_id: 1} index, and takes a very long time to run, while greatly increasing the mongo servers cpu usage.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Here are the query execution stats for a datasetId where ~ 40K documents satisfy the query, and an _id which is the oldest available that has specifed datasetId - which took 28.6 seconds:&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/394777/394777_image-2022-08-09-16-24-30-226.png&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/394778/394778_image-2022-08-09-16-23-38-511.png&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;If I hint the index on the same query it uses it and runs very quickly scanning only the needed documents:&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/394774/394774_image-2022-08-09-16-32-59-723.png&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;Using any other operator than $gt /$gte works ($lt / $lte on the newest available ObjectId, $ne - all use the index with no need for a hint), the greater than operators are the only ones that do not use the index (example of a $lt query):&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/394773/394773_image-2022-08-09-16-34-22-817.png&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/394772/394772_image-2022-08-09-16-36-06-516.png&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;I cannot hint at the index each in query of this nature, Why do only the $gt and $gte operators not use the index? seems they should act like any other operator and use the index if it is available?&lt;/p&gt;</description>
                <environment></environment>
        <key id="2109498">SERVER-68672</key>
            <summary>Query using shard key and document ID does not use available index</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="eric.sedor@mongodb.com">Eric Sedor</assignee>
                                    <reporter username="shachar.h@dataloop.ai">Shachar Hochberg</reporter>
                        <labels>
                    </labels>
                <created>Tue, 9 Aug 2022 13:36:13 +0000</created>
                <updated>Tue, 30 Aug 2022 20:37:44 +0000</updated>
                            <resolved>Tue, 30 Aug 2022 20:37:44 +0000</resolved>
                                    <version>4.2.21</version>
                                                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="4745404" author="eric.sedor" created="Thu, 11 Aug 2022 20:31:34 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=shachar.h%40dataloop.ai&quot; class=&quot;user-hover&quot; rel=&quot;shachar.h@dataloop.ai&quot;&gt;shachar.h@dataloop.ai&lt;/a&gt;, this looks very similar to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-17260&quot; title=&quot;Wrong index is often chosen for query: { &amp;quot;foo&amp;quot;: &amp;quot;fooValue&amp;quot;, &amp;quot;_id&amp;quot;: { &amp;quot;$gte&amp;quot;: &amp;quot;idValue&amp;quot; } }, sort: { &amp;quot;_id&amp;quot;: 1 }&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-17260&quot;&gt;&lt;del&gt;SERVER-17260&lt;/del&gt;&lt;/a&gt; which is considered a duplicate of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-15225&quot; title=&quot;CachedPlanStage should execute for trial period and re-plan if query performs poorly&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-15225&quot;&gt;&lt;del&gt;SERVER-15225&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Can you revisit your use-case in the context of the configuration option and the workarounds discussed on &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-15225&quot; title=&quot;CachedPlanStage should execute for trial period and re-plan if query performs poorly&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-15225&quot;&gt;&lt;del&gt;SERVER-15225&lt;/del&gt;&lt;/a&gt;, and let us know how it goes?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="157798">SERVER-15225</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="394780" name="image-2022-08-09-13-42-22-395.png" size="526772" author="shachar.h@dataloop.ai" created="Tue, 9 Aug 2022 10:42:24 +0000"/>
                            <attachment id="394779" name="image-2022-08-09-16-23-11-039.png" size="44306" author="shachar.h@dataloop.ai" created="Tue, 9 Aug 2022 13:23:12 +0000"/>
                            <attachment id="394778" name="image-2022-08-09-16-23-38-511.png" size="28254" author="shachar.h@dataloop.ai" created="Tue, 9 Aug 2022 13:23:39 +0000"/>
                            <attachment id="394777" name="image-2022-08-09-16-24-30-226.png" size="44301" author="shachar.h@dataloop.ai" created="Tue, 9 Aug 2022 13:24:31 +0000"/>
                            <attachment id="394776" name="image-2022-08-09-16-32-04-424.png" size="18447" author="shachar.h@dataloop.ai" created="Tue, 9 Aug 2022 13:32:05 +0000"/>
                            <attachment id="394775" name="image-2022-08-09-16-32-24-886.png" size="18892" author="shachar.h@dataloop.ai" created="Tue, 9 Aug 2022 13:32:25 +0000"/>
                            <attachment id="394774" name="image-2022-08-09-16-32-59-723.png" size="15100" author="shachar.h@dataloop.ai" created="Tue, 9 Aug 2022 13:33:00 +0000"/>
                            <attachment id="394773" name="image-2022-08-09-16-34-22-817.png" size="15146" author="shachar.h@dataloop.ai" created="Tue, 9 Aug 2022 13:34:23 +0000"/>
                            <attachment id="394772" name="image-2022-08-09-16-36-06-516.png" size="17274" author="shachar.h@dataloop.ai" created="Tue, 9 Aug 2022 13:36:07 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 11 Aug 2022 20:31:34 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 25 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>eric.sedor@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 25 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>eric.sedor@mongodb.com</customfieldvalue>
            <customfieldvalue>shachar.h@dataloop.ai</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i15kzb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i0od8o:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                    <customfieldvalue><![CDATA[eric.sedor@mongodb.com]]></customfieldvalue>
    

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1574n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>