<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:54:18 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-62150] SBE Multiplanning can be slow when suboptimal plan runs first</title>
                <link>https://jira.mongodb.org/browse/SERVER-62150</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Currently, the strategy used in SBE multiplanning is as follows:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;We run non blocking plans before blocking ones.&lt;/li&gt;
	&lt;li&gt;We run each plan&#8217;s trial period to completion (i.e. until we return 101 documents or we use up the plans budget). We use the number of reads performed by said plan to bound the number of reads used by any remaining plans.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;The problem with this approach is that if the first plan we run is not the optimal one, we are stuck running it and we can potentially use all of the reads. As an example, consider two plans, A and B. Plan A needs to perform 10k storage engine reads to get 101 documents, while plan B needs to perform 101 reads to get 101 documents. If Plan B runs first, we have no problems: we will set the reads limit for plan A to 101, and it will stop running after 101 reads. If Plan A runs first however, we will be stuck running plan A for all 10k reads.&#160; Though we&#8217;ll eventually run plan B and it will be chosen, this negatively impacts the performance of queries which need to use the multiplanner.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1953500">SERVER-62150</key>
            <summary>SBE Multiplanning can be slow when suboptimal plan runs first</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="1" iconUrl="https://jira.mongodb.org/images/icons/statuses/open.png" description="">Open</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-query-execution">Backlog - Query Execution</assignee>
                                    <reporter username="mihai.andrei@mongodb.com">Mihai Andrei</reporter>
                        <labels>
                            <label>RDY</label>
                    </labels>
                <created>Fri, 17 Dec 2021 15:53:04 +0000</created>
                <updated>Sat, 27 Jan 2024 09:02:00 +0000</updated>
                                            <version>5.1.1</version>
                    <version>5.2.0-rc1</version>
                    <version>6.0.12</version>
                    <version>7.0.4</version>
                                                                        <votes>3</votes>
                                    <watches>60</watches>
                                                                                                                <comments>
                            <comment id="5918608" author="JIRAUSER1254095" created="Fri, 1 Dec 2023 16:26:06 +0000"  >&lt;p&gt;David, Ivan thank you both it means a lot to me as a customer that MongoDB is tackling these issues with high priority.&lt;/p&gt;</comment>
                            <comment id="5917174" author="david.storch" created="Fri, 1 Dec 2023 00:01:16 +0000"  >&lt;p&gt;To add onto what Ivan said, I&apos;m going to move this ticket back to the &quot;Open&quot; state &#8211; we are no longer &quot;Investigating&quot; this issue, but rather are executing on an engineering project to fix the problem. The solution will be delivered against a sequence of related Jira tickets rather than developing directly against this ticket. However, we can provide high-level progress updates here.&lt;/p&gt;</comment>
                            <comment id="5915015" author="JIRAUSER1270969" created="Thu, 30 Nov 2023 11:42:21 +0000"  >&lt;p&gt;Yes, I am aware. &lt;/p&gt;

&lt;p&gt;We are working on a solution. However, it requires redesign of the whole multi planning process with SBE and will take some time to develop, test and release.&lt;/p&gt;

&lt;p&gt;To improve customer experience in the meantime, we are planning a change in default configuration via &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-83470&quot; title=&quot;Introduce internalQueryFrameworkControl setting for 6.0-style engine selection logic&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-83470&quot;&gt;&lt;del&gt;SERVER-83470&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;As well we are going to improve our testing process not to miss this again with the next SBE release.&lt;/p&gt;</comment>
                            <comment id="5914985" author="JIRAUSER1254095" created="Thu, 30 Nov 2023 11:25:31 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=ivan.fefer%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;ivan.fefer@mongodb.com&quot;&gt;ivan.fefer@mongodb.com&lt;/a&gt; are you also aware of this issue? This is another one we are seeing related to SBE.&lt;/p&gt;</comment>
                            <comment id="5865337" author="JIRAUSER1254095" created="Wed, 8 Nov 2023 19:14:03 +0000"  >&lt;p&gt;Following from my report in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-82549&quot; title=&quot;MongoDB 7.0.2 SBE query slow when direct index exists&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-82549&quot;&gt;&lt;del&gt;SERVER-82549&lt;/del&gt;&lt;/a&gt;, I&apos;d like to underscore how app-breaking and frustrating this issue is in MongoDB 7. (We didn&apos;t see any effects in Mongo 5 or 6.)&lt;/p&gt;

&lt;p&gt;I anticipate many CRUD apps on MongoDB will be affected by this. When we initially upgraded to MongoDB 7 ~3 weeks ago we saw it breaking our app in a number of critical places. Please give this issue the attention it deserves.&lt;/p&gt;</comment>
                            <comment id="4477213" author="JIRAUSER1257467" created="Tue, 12 Apr 2022 15:08:46 +0000"  >&lt;p&gt;Waiting on other tickets first&lt;/p&gt;</comment>
                            <comment id="4465893" author="david.storch" created="Thu, 7 Apr 2022 20:41:47 +0000"  >&lt;p&gt;Returning this to the triage queue. At the moment our efforts related to this problem fall under &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-63642&quot; title=&quot;Add serverStatus metrics to measure multi-planning performance&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-63642&quot;&gt;&lt;del&gt;SERVER-63642&lt;/del&gt;&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-63641&quot; title=&quot;Improve SBE multi-planning by choosing which plan to work next based on a priority metric&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-63641&quot;&gt;&lt;del&gt;SERVER-63641&lt;/del&gt;&lt;/a&gt;, so there is no action for me currently planned against this umbrella ticket.&lt;/p&gt;</comment>
                            <comment id="4354929" author="david.storch" created="Mon, 14 Feb 2022 23:19:22 +0000"  >&lt;p&gt;Another quick update. We have filed two additional offshoot tickets:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-63642&quot; title=&quot;Add serverStatus metrics to measure multi-planning performance&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-63642&quot;&gt;&lt;del&gt;SERVER-63642&lt;/del&gt;&lt;/a&gt; &quot;Add serverStatus metrics to measure multi-planning performance&quot;. The work for this ticket would add telemetry to help us to understand the performance of the SBE multi-planner across the Atlas fleet.&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-63641&quot; title=&quot;Improve SBE multi-planning by choosing which plan to work next based on a priority metric&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-63641&quot;&gt;&lt;del&gt;SERVER-63641&lt;/del&gt;&lt;/a&gt; &quot;Improve SBE multi-planning by choosing which plan to work next based on a priority metric&quot;. This ticket tracks the improvement to the SBE multi-planning algorithm proposed by &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=mihai.andrei&quot; class=&quot;user-hover&quot; rel=&quot;mihai.andrei&quot;&gt;mihai.andrei&lt;/a&gt; which I have already summarized above. The ticket description contains a more detailed writeup of the proposed change. This work could help to improve the performance of SBE multi-planning beyond what was already achieved in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-62981&quot; title=&quot;Make SBE multi-planner&amp;#39;s trial period termination condition independent of collection size&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-62981&quot;&gt;&lt;del&gt;SERVER-62981&lt;/del&gt;&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Folks interested in this ticket may wish to watch these two new related ones. This ticket will continue to serve as the umbrella. There is no specific engineering work planned against the umbrella ticket at this time, but &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-63642&quot; title=&quot;Add serverStatus metrics to measure multi-planning performance&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-63642&quot;&gt;&lt;del&gt;SERVER-63642&lt;/del&gt;&lt;/a&gt; is scheduled and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-63641&quot; title=&quot;Improve SBE multi-planning by choosing which plan to work next based on a priority metric&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-63641&quot;&gt;&lt;del&gt;SERVER-63641&lt;/del&gt;&lt;/a&gt; will be triaged by the Query Execution team.&lt;/p&gt;</comment>
                            <comment id="4321702" author="david.storch" created="Fri, 28 Jan 2022 18:13:00 +0000"  >&lt;p&gt;Related ticket &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-62981&quot; title=&quot;Make SBE multi-planner&amp;#39;s trial period termination condition independent of collection size&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-62981&quot;&gt;&lt;del&gt;SERVER-62981&lt;/del&gt;&lt;/a&gt; has now been completed for versions 5.3.0 and 5.2.1 which we anticipate will help a lot with the problem described by this ticket.&lt;/p&gt;</comment>
                            <comment id="4314517" author="david.storch" created="Tue, 25 Jan 2022 23:58:57 +0000"  >&lt;p&gt;The Query Team has been internally brainstorming several potential solutions to this problem. We have generated a handful of ideas of various implementation complexity which I will describe below, mostly for the benefit of query engineering. However, we think there is one simple change that we should implement immediately, which we expect should go a long way towards mitigating the problem described here:&#160;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-62981&quot; title=&quot;Make SBE multi-planner&amp;#39;s trial period termination condition independent of collection size&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-62981&quot;&gt;&lt;del&gt;SERVER-62981&lt;/del&gt;&lt;/a&gt;. I suggest that folks interested in this ticket also watch&#160;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-62981&quot; title=&quot;Make SBE multi-planner&amp;#39;s trial period termination condition independent of collection size&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-62981&quot;&gt;&lt;del&gt;SERVER-62981&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Once &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-62981&quot; title=&quot;Make SBE multi-planner&amp;#39;s trial period termination condition independent of collection size&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-62981&quot;&gt;&lt;del&gt;SERVER-62981&lt;/del&gt;&lt;/a&gt; is complete, we could consider pursuing one of the following additional changes in the future:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=mihai.andrei&quot; class=&quot;user-hover&quot; rel=&quot;mihai.andrei&quot;&gt;mihai.andrei&lt;/a&gt;&apos;s idea: during SBE multi-planning, always call &lt;tt&gt;getNext()&lt;/tt&gt; on whichever plan currently seems the most promising. This shouldn&apos;t be too hard to implement, but it still suffers from the problem where a single call to &lt;tt&gt;getNext()&lt;/tt&gt; for an unselective plan could expend the entire reads budget.&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=christopher.harris&quot; class=&quot;user-hover&quot; rel=&quot;christopher.harris&quot;&gt;christopher.harris&lt;/a&gt; points out that we could use the classic multi-planner for plan selection, but then hand the winning plan off to the SBE engine for execution. In this scheme, we would continue to use SBE when recovering plans from the plan cache. One downside of this approach is that any partial results computed during the trial period would have to be completely thrown out, and execution started from the beginning in SBE. It would also be quite complex to implement. However, it would mean that enabling SBE would not change the behavior of the system with regards to plan selection. It would also give us an opportunity to restore some of the more useful aspects of the explain format at &quot;allPlansExecution&quot; verbosity.&lt;/li&gt;
	&lt;li&gt;I propose a two-phase approach in the SBE multi-planner. The first round would run the trial plan for each candidate much like the SBE multi-planner&apos;s current process, but it would use a much smaller reads budget. For instance, this reads budget could be on the order of 100 or 200. If any plan hits EOF or produces its first batch of results, then a winner is chosen according to the current ranking formula. Otherwise, we move onto the second round using the much larger reads budget of 10,000. The idea is to make sure that multi-planning terminates as quickly as possible without exploring bad candidate plans when one of the available candidates is very cheap.&lt;/li&gt;
&lt;/ul&gt;
</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="2485079">SERVER-82549</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10520">
                    <name>Problem/Incident</name>
                                            <outwardlinks description="causes">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="2488357">SERVER-82677</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1970971">SERVER-62981</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1973289">SERVER-63102</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1983353">SERVER-63642</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1983346">SERVER-63641</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="2498731">SERVER-83196</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>10.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25125"><![CDATA[Query Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[5002K000010EZueQAG, 5002K000010GWfRQAW, 5002K000010IQulQAG, 5002K000011BtDpQAK, 5002K000011BUjBQAW, 5002K000011BPEgQAO, 5002K000011D6HxQAK, 5002K000011gOtQQAU, 5002K000011CaExQAK, 5006R00001ssozVQAQ, 5006R00001y6DYhQAM]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 21 Dec 2021 15:48:49 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        9 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/TSEXP-2028'>TSEXP-2028</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-3591</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            9 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>ana.meza@mongodb.com</customfieldvalue>
            <customfieldvalue>backlog-query-execution</customfieldvalue>
            <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
            <customfieldvalue>ivan.fefer@mongodb.com</customfieldvalue>
            <customfieldvalue>shields@tablecheck.com</customfieldvalue>
            <customfieldvalue>mihai.andrei@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0f2z3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr29tb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="5810">QE 2022-01-24</customfieldvalue>
    <customfieldvalue id="7258">QO 2023-11-13</customfieldvalue>
    <customfieldvalue id="7260">QO 2023-11-27</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0ep4f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>