<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:49:52 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-19114] Rooted $or queries with a sort may select a SORT plan over a potentially faster SORT_MERGE plan</title>
                <link>https://jira.mongodb.org/browse/SERVER-19114</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;div class=&quot;panel&quot; style=&quot;background-color: #eeeeee;border-color: #cccccc;border-width: 1px;&quot;&gt;&lt;div class=&quot;panelHeader&quot; style=&quot;border-bottom-width: 1px;border-bottom-color: #cccccc;background-color: #6cb33f;&quot;&gt;&lt;b&gt;Issue Status as of March 9, 2020&lt;/b&gt;&lt;/div&gt;&lt;div class=&quot;panelContent&quot; style=&quot;background-color: #eeeeee;&quot;&gt;
&lt;p&gt;&lt;b&gt;ISSUE DESCRIPTION AND IMPACT&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;Queries consisting of a single $or element at the top level of the query JSON are called &quot;rooted $or&quot; queries. For example:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;{$or: [{a: 1}, {b: 1}]} is a rooted $or query&lt;/li&gt;
	&lt;li&gt;{c:1, $or: [{a: 1}, {b: 1}]} is not&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;When a rooted $or query includes a sort specification, the MongoDB query planner can choose a sub-optimal query plan. This is because the planner evaluates each $or clause independently and doesn&apos;t evaluate each clause with the knowledge that they are being sorted in the same way.&lt;/p&gt;

&lt;p&gt;As a result, one or more $or clauses may use indexes that do not provide results in the order required by the sort, necessitating a potentially slow blocking SORT stage in the execution plan.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;DIAGNOSIS AND AFFECTED VERSIONS&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;This issue can be confirmed using explain(). When a blocking sort is being used, a SORT stage will be present (instead of SORT_MERGE) in the &lt;a href=&quot;https://docs.mongodb.com/manual/reference/method/cursor.explain/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;.explain(&quot;queryPlanner&quot;)&lt;/a&gt; output.&lt;/p&gt;

&lt;p&gt;This issue affects all currently supported versions of MongoDB.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;REMEDIATION AND WORKAROUNDS&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;For cases where all $or clauses can use the same index, one possible workaround is to use &lt;a href=&quot;https://docs.mongodb.com/manual/reference/method/cursor.hint/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;hint&lt;/a&gt; to that query to force a merge sort.&#160;The presence of a SORT_MERGE stage in the explain output indicates that a merge sort is being used. The plan chosen for each clause of the $or is producing the requested sort order, and a total order can be obtained by merging the results from each sorted stream.&#160;&lt;/p&gt;

&lt;p&gt;Another possible workaround&#160;to force a merge sort is to constrain index consideration using an &lt;a href=&quot;https://docs.mongodb.com/manual/core/query-plans/#index-filters&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;index filter&lt;/a&gt;. Note that index filters do not persist across server restarts and must be configured individually on each node in the cluster.&lt;/p&gt;

&lt;p&gt;Additionally, It is necessary to ensure:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Indexes for each $or clause exist and provide results in the desired sort order.&lt;/li&gt;
	&lt;li&gt;There are no indexes for each $or clause that would require a blocking sort to satisfy the desired sort order.&lt;/li&gt;
&lt;/ul&gt;
&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;Rooted $or queries are planned by a special &quot;subplanner&quot; mechanism in which the best plan for each $or clause is selected individually. For rooted $or queries with a sort, the subplanner&apos;s &quot;local&quot; optimization of each clause may result in a suboptimal global query plan.&lt;/p&gt;

&lt;p&gt;In particular, the subplanner can cause us to choose a plan with a blocking SORT stage, entirely missing the fact that a non-blocking SORT_MERGE plan is available. Suppose that there is a rooted $or query with two clauses and a sort:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;db.collection.find({$or: [clause1, clause2]}).sort(sortSpec);&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;Suppose that there are two plans for &lt;em&gt;clause1&lt;/em&gt;:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;&lt;em&gt;plan1&lt;/em&gt;, which provides the sort &lt;em&gt;sortSpec&lt;/em&gt;, and&lt;/li&gt;
	&lt;li&gt;&lt;em&gt;plan2&lt;/em&gt;, which does not provide the sort &lt;em&gt;sortSpec&lt;/em&gt;.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;If &lt;em&gt;plan2&lt;/em&gt; is a better plan for &lt;em&gt;clause1&lt;/em&gt;, then the subplanner will always select it. However, if the winning plan for &lt;em&gt;clause2&lt;/em&gt; provides the sort &lt;em&gt;sortSpec&lt;/em&gt;, then it may actually be better to select &lt;em&gt;plan1&lt;/em&gt; for &lt;em&gt;clause1&lt;/em&gt;. Selecting the plan that provides the sort means that both subplans provide the sort, and therefore they can be merged using the merge phase of the mergeSort algorithm, rather than performing a full in-memory sort. The subplanner does not currently evaluate this tradeoff, but rather always chooses &lt;em&gt;plan2&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;See repro steps for a simple script to reproduce.&lt;/p&gt;</description>
                <environment></environment>
        <key id="212787">SERVER-19114</key>
            <summary>Rooted $or queries with a sort may select a SORT plan over a potentially faster SORT_MERGE plan</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-query-optimization">Backlog - Query Optimization</assignee>
                                    <reporter username="david.storch@mongodb.com">David Storch</reporter>
                        <labels>
                            <label>query-44-grooming</label>
                    </labels>
                <created>Wed, 24 Jun 2015 17:43:22 +0000</created>
                <updated>Tue, 6 Dec 2022 04:49:28 +0000</updated>
                                                                            <component>Querying</component>
                                        <votes>2</votes>
                                    <watches>29</watches>
                                                                                                                <comments>
                            <comment id="4201450" author="JIRAUSER1257467" created="Fri, 19 Nov 2021 18:43:35 +0000"  >&lt;p&gt;Reopening for reconsideration, maybe as part of CQF&lt;/p&gt;

&lt;p&gt;This one was part of &#160;PM-416&#160;&#160;but the EPIC was closed as Won&apos;t do&lt;/p&gt;</comment>
                            <comment id="1105894" author="sofavezi@momentfeed.com" created="Fri, 4 Dec 2015 19:26:08 +0000"  >&lt;p&gt;We are hitting this problem and we really need this to get resolved asap. I&apos;m sure other Mongo users have the same problem as this is a problem with plan engine and this is core to mongo and any user is relying on the plan engine to work properly.&lt;br/&gt;
Can you please move this ticket to high priority? &lt;/p&gt;
</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="942758">SERVER-43671</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25126"><![CDATA[Query Optimization]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[500A000000Zq4yIIAR, 500A000000bzanvIAA, 5002K00000ej7UxQAI, 5002K00000r4c31QAA]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 4 Dec 2015 19:26:08 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 11 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 11 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>ana.meza@mongodb.com</customfieldvalue>
            <customfieldvalue>backlog-query-optimization</customfieldvalue>
            <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
            <customfieldvalue>sofavezi@momentfeed.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrl2en:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr2icf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;db.coll.drop();&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;db.coll.ensureIndex({a: 1});&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;db.coll.ensureIndex({b: 1});&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;for (var i = 0; i &amp;lt; 10; i++) { db.coll.insert({b: 1}); }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;// A SORT plan will be selected. The SORT_MERGE plan is not considered.&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;db.coll.explain(&quot;queryPlanner&quot;).find({$or: [{a: 1, b: 1}, {b: 1}]}).sort({b: 1});&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsfwr3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>