<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:32:44 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-13740] Improve plan ranking for index intersection</title>
                <link>https://jira.mongodb.org/browse/SERVER-13740</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We know from performance testing that index intersection plans are faster than single-solution plans when&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;The size of the intersection is small.&lt;/li&gt;
	&lt;li&gt;The amount of data that would need to be fetched by the single-index solutions is large, so that&lt;/li&gt;
	&lt;li&gt;this data would exceed the available memory, or is generally not in the working set.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;However, our testing also shows that index intersection plans may not be chosen by the ranker in these cases. We should consider tweaking plan ranking such that the ranker is more likely to choose intersection plans.&lt;/p&gt;</description>
                <environment></environment>
        <key id="133380">SERVER-13740</key>
            <summary>Improve plan ranking for index intersection</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-query-optimization">Backlog - Query Optimization</assignee>
                                    <reporter username="david.storch@mongodb.com">David Storch</reporter>
                        <labels>
                            <label>bonsai</label>
                    </labels>
                <created>Fri, 25 Apr 2014 17:43:55 +0000</created>
                <updated>Tue, 6 Dec 2022 05:06:59 +0000</updated>
                                            <version>2.6.0</version>
                                                    <component>Querying</component>
                                        <votes>11</votes>
                                    <watches>33</watches>
                                                                                                                <comments>
                            <comment id="3498169" author="JIRAUSER1257697" created="Thu, 19 Nov 2020 18:18:12 +0000"  >&lt;p&gt;I&apos;d like to add that my company has recently run into this issue.&#160; I&apos;d like to request a change in the documentation.&#160;&#160;&lt;a href=&quot;https://docs.mongodb.com/manual/core/index-intersection/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://docs.mongodb.com/manual/core/index-intersection/&lt;/a&gt;&#160;&lt;/p&gt;

&lt;p&gt;&quot;MongoDB can use the intersection of multiple indexes to fulfill queries. In general, each index intersection involves two indexes; however, MongoDB can employ multiple/nested index intersections to resolve a query.&quot;&lt;/p&gt;

&lt;p&gt;is very misleading.&#160; The query planner is not able to make use of multiple indexes in the majority of our use cases, which is pretty disappointing given the documentation.&#160; The ability to hint at index intersection would be a good solution for us.&lt;/p&gt;</comment>
                            <comment id="3026743" author="kreig" created="Mon, 6 Apr 2020 04:10:47 +0000"  >&lt;p&gt;In case of large collections with billions of documents indexes intersection may potentially provide a predictable performance, which is far more important than possible minor performance gains from the IXSCAN + filtered FETCH plan.&lt;/p&gt;

&lt;p&gt;We&apos;ve built a dozen of projects on top of MongoDB during the past decade, and this query planner issue is a huge pain for us. It&apos;s like a nightmare that keeps haunting us for years. A project is ready, deployed on production, everything works flawlessly and everybody is immensely happy with the performance. But two months later the database grows to a several millions of documents and some queries start slowing down. We build compound indexes on 4-6 fields that participate in query, set hints in code and forget about it for a few more months. Nevertheless, sooner or later the compound index grows to hundreds of gigabytes and it doesn&apos;t fit in RAM anymore. Simple intersection of 3 or 4 small indexes would do the trick, but there is no way to specify the hint or adjust the query. And we have to migrate everything to another db engine even despite the fact that 98% of all queries will be much slower than before, but usually it&apos;s those 2% left that really matter, because they may easily result in the completely degraded performance for the entire application. The situation repeats again and again. We had to migrate 3 of our large projects so far, and it seems like we&apos;ll have to do the same with two more applications in the nearest future.&#160;There was even a situation when we moved a legacy system from Oracle to MongoDB just to migrate it back again three years later when the compound indexes performance plunged due to their enormous size.&lt;/p&gt;

&lt;p&gt;That&apos;s especially painful to me, as Mongo was my favorite db engine for years. However, having predictable performance for all queries allows us to plan future scalability, so naturally, it&apos;s the paramount consideration.&#160;MongoDB works flawlessly with relatively small datasets or simple queries, so it&apos;s definitely our primary choice for prototyping and small projects.&lt;/p&gt;

&lt;p&gt;But I really hope that you prioritize this problem and include the fix in the upcoming major release, because right now we simply can&apos;t achieve reasonable query processing time in two of our ongoing projects. Judging from the numerous questions StackOverflow, many devs have the same issue. Please, consider prioritizing this or at least allow hinting the intersection manually (&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-22741&quot; title=&quot;Ability to hint index intersection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-22741&quot;&gt;SERVER-22741&lt;/a&gt;) - this will do the trick for the most severe cases.&lt;/p&gt;

&lt;p&gt;As a side note, index intersection plan doesn&apos;t even appear in the rejected plans in the following cases:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;one of the filter conditions is not an equality operator&#160;&lt;font color=&quot;#4c9aff&quot;&gt;&lt;tt&gt;{a: 1, b: \{$gt: 1}&lt;/tt&gt;}&lt;/font&gt;&lt;/li&gt;
	&lt;li&gt;one of the indexes is a multikey index &lt;font color=&quot;#4c9aff&quot;&gt;&lt;tt&gt;{intField: 1, arrayField: 1}&lt;/tt&gt;&lt;/font&gt;&lt;/li&gt;
	&lt;li&gt;with more than 3 conditions &lt;tt&gt;&lt;font color=&quot;#4c9aff&quot;&gt;{a: 1, b: 1, c: 1, d: 1}&lt;/font&gt;&lt;/tt&gt;&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;(checked on 4.2.1)&lt;/p&gt;</comment>
                            <comment id="942895" author="wrichard" created="Wed, 17 Jun 2015 13:42:02 +0000"  >&lt;p&gt;This issue is especially bad when using the WiredTiger storage engine, as the scan over the single-index solution can require decompression of very large amounts of data (not just a quick scan over small parts of memory mapped files). The script attached to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-17821&quot; title=&quot;Query planner prefers suboptimal index to using index intersection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-17821&quot;&gt;&lt;del&gt;SERVER-17821&lt;/del&gt;&lt;/a&gt; replicates this issue. This issue is preventing us from migrating to WiredTiger due to a ~1000x performance loss.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="156136">SERVER-15125</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="193211">SERVER-17821</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="370430">SERVER-28587</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="231398">SERVER-20619</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25126"><![CDATA[Query Optimization]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[5002K00000pFvUEQA0, 5002K00000plP44QAE]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 17 Jun 2015 13:42:02 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 11 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 11 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-query-optimization</customfieldvalue>
            <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
            <customfieldvalue>patrick@checkr.com</customfieldvalue>
            <customfieldvalue>kreig</customfieldvalue>
            <customfieldvalue>wrichard</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlw1r:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr2ljz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>114612</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrxglz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>