<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:56:11 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-21064] It&apos;s possible to fail on sort memory overflow even if there is an alternative plan that requires no sort.</title>
                <link>https://jira.mongodb.org/browse/SERVER-21064</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;With a collection that has two indexes like:&lt;br/&gt;
1) a_1_b_1_c_1&lt;br/&gt;
2) c_1_b_1_a_1&lt;/p&gt;

&lt;p&gt;If you run a query like: {a: { &quot;$in&quot;: &lt;span class=&quot;error&quot;&gt;&amp;#91;1&amp;#93;&lt;/span&gt;}, b:1, c: { &quot;$in&quot;: &lt;span class=&quot;error&quot;&gt;&amp;#91;1,2,3&amp;#93;&lt;/span&gt;}}).sort({c:1}), a query shape will be cached with the associated plans that do no require sort and it is likely that the plan using the a_1_b_1_c_1 will get the top score.&lt;/p&gt;

&lt;p&gt;If you now run  query like: {a: { &quot;$in&quot;: &lt;span class=&quot;error&quot;&gt;&amp;#91;1,2,3&amp;#93;&lt;/span&gt;}, b:1, c: { &quot;$in&quot;: &lt;span class=&quot;error&quot;&gt;&amp;#91;1,2,3&amp;#93;&lt;/span&gt;}}).sort({c:1}), the cached plan with top score will be used. As the query requires SORT stage, the plan will be re-adjusted, however if at the sort stage we hit the 32Mb limit, the query will fail despite the fact that there is an alternative plan cached that uses the c_1_b_1_a_1 index, which if used, will require no sort and re-adjustment.&lt;/p&gt;</description>
                <environment></environment>
        <key id="235625">SERVER-21064</key>
            <summary>It&apos;s possible to fail on sort memory overflow even if there is an alternative plan that requires no sort.</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="david.storch@mongodb.com">David Storch</assignee>
                                    <reporter username="dmitry.ryabtsev@mongodb.com">Dmitry Ryabtsev</reporter>
                        <labels>
                    </labels>
                <created>Thu, 22 Oct 2015 04:11:00 +0000</created>
                <updated>Fri, 23 Oct 2015 14:16:09 +0000</updated>
                            <resolved>Fri, 23 Oct 2015 14:16:09 +0000</resolved>
                                    <version>3.0.3</version>
                                                    <component>Querying</component>
                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="1068919" author="david.storch" created="Fri, 23 Oct 2015 14:15:12 +0000"  >&lt;p&gt;After reproducing the issue, I can confirm that this is a duplicate of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-15225&quot; title=&quot;CachedPlanStage should execute for trial period and re-plan if query performs poorly&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-15225&quot;&gt;&lt;del&gt;SERVER-15225&lt;/del&gt;&lt;/a&gt;. Please read on for a more detailed description of the issue.&lt;/p&gt;

&lt;p&gt;Prior to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-15225&quot; title=&quot;CachedPlanStage should execute for trial period and re-plan if query performs poorly&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-15225&quot;&gt;&lt;del&gt;SERVER-15225&lt;/del&gt;&lt;/a&gt;, the CachedPlanStage had a notion of a &quot;backup plan&quot;. Specifically, whenever a plan with a blocking in-memory SORT stage was added to the plan cache, we would look for a non-blocking plan to act as &quot;backup&quot; (&lt;a href=&quot;https://github.com/mongodb/mongo/blob/aca53a0693d50bc33d7a8c6d3548795a00334bdc/src/mongo/db/query/plan_cache.cpp#L329-L338&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;see the code here&lt;/a&gt;). If the plan is taken from the cache on a subsequent execution of the query shape, and the cached plan fails by hitting the 32 MB limit for in-memory sorts, the CachedPlanStage would fall back on the backup plan.&lt;/p&gt;

&lt;p&gt;This ticket describes a scenario in which the cached plan backup plan strategy did not work. Suppose you have index {a: 1, b: 1, c: 1}. Consider plans which answer the following two predicates by scanning this index:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;{a: { &quot;$in&quot;: [1]}, b:1, c: { &quot;$in&quot;: [1,2,3]}}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;{a: { &quot;$in&quot;: [1,2,3]}, b:1, c: { &quot;$in&quot;: [1,2,3]}}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;The former scan will be sorted by {c: 1} since it will return documents where &lt;tt&gt;c&lt;/tt&gt; is 1, followed by docs where &lt;tt&gt;c&lt;/tt&gt; is 2, followed by docs where &lt;tt&gt;c&lt;/tt&gt; is 3. On the other hand, the latter scan will not be sorted by {c: 1}. However, as reported in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21065&quot; title=&quot;Optimizer should consider predicate with &amp;quot;$in&amp;quot; operator having a single argument to be equivalent to the equality operator&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21065&quot;&gt;&lt;del&gt;SERVER-21065&lt;/del&gt;&lt;/a&gt;, these two predicates are considered to have the same shape. (The number of &lt;tt&gt;$in&lt;/tt&gt; elements is not considered significant when computing query shape.) This leads to the following situation:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;The client issues a query with predicate {a: { &quot;$in&quot;: [1]}, b:1, c: { &quot;$in&quot;: [1,2,3]}} and sort {c: 1}.&lt;/li&gt;
	&lt;li&gt;The non-blocking plan which obtains the sort order by scanning index {a: 1, b: 1, c: 1} is selected as the winner and cached. Since the plan has no SORT stage, no backup plan is incorporated into the plan cache entry.&lt;/li&gt;
	&lt;li&gt;The client issues a query with predicate {a: { &quot;$in&quot;: [1,2,3]}, b:1, c: { &quot;$in&quot;: [1,2,3]}} and sort {c: 1}.&lt;/li&gt;
	&lt;li&gt;This query is considered the same shape as the prior one, so the query will used the cached plan. The plan building process correctly identifies that the index scan is not providing the sort on {c: 1}, and adds a SORT stage.&lt;/li&gt;
	&lt;li&gt;The in-memory sort hits the 32 MB limit. However, the cache entry has no backup plan, so rather than attempting to fall back on a non-blocking plan, an error is returned to the application.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;This is fixed by &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-15225&quot; title=&quot;CachedPlanStage should execute for trial period and re-plan if query performs poorly&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-15225&quot;&gt;&lt;del&gt;SERVER-15225&lt;/del&gt;&lt;/a&gt;, which eliminated the notion of a cached &quot;backup plan&quot; and replaced it with the notion of &quot;replanning&quot;. When this scenario occurs on a version where replanning is enabled by default (3.0.7 and higher), the cached plan hitting the 32 MB limit will cause us to restart plan selection from the beginning. We generate and rank several candidate plans, and correctly select the non-blocking plan as the winner.&lt;/p&gt;</comment>
                            <comment id="1068008" author="david.storch" created="Thu, 22 Oct 2015 16:23:07 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=dmitry.ryabtsev&quot; class=&quot;user-hover&quot; rel=&quot;dmitry.ryabtsev&quot;&gt;dmitry.ryabtsev&lt;/a&gt;, do have a repro script you could share? I tried to write a script based on your description but wasn&apos;t able to reproduce. Your repro would help me confirm that this issue was fixed by &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-15225&quot; title=&quot;CachedPlanStage should execute for trial period and re-plan if query performs poorly&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-15225&quot;&gt;&lt;del&gt;SERVER-15225&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="1067604" author="dmitry.ryabtsev" created="Thu, 22 Oct 2015 05:50:25 +0000"  >&lt;p&gt;On 3.0.7 I&apos;m able to reproduce this only after &quot;db.runCommand({setParameter: 1, internalQueryCacheReplanningEnabled: false})&quot;. My understanding is that we re-adjust the cached plan having the top score by adding sort stage. Then we score it and compare the score against the second plan we have which has the optimal index. I&apos;m concerned that this is a workaround but not the fix, as the winning plan can be the plan with sort.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="157798">SERVER-15225</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="235630">SERVER-21065</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="95263" name="template" size="39985" author="dmitry.ryabtsev@mongodb.com" created="Thu, 22 Oct 2015 23:28:04 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 22 Oct 2015 16:23:07 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        8 years, 16 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>david.storch@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            8 years, 16 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
            <customfieldvalue>dmitry.ryabtsev@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrkr2f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hsdvc7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="740">QuInt B (11/02/15)</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsfj7z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>