<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:01:59 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-3071] Index Intersection</title>
                <link>https://jira.mongodb.org/browse/SERVER-3071</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Update:&lt;/p&gt;

&lt;p&gt;Index intersection is on by default.  The query planner currently limits itself to 2 indices at a time, though this limit may increase in the future.&lt;/p&gt;



&lt;p&gt;Original Text:&lt;/p&gt;

&lt;p&gt;This should support both single and multiple index intersections.&lt;/p&gt;

&lt;p&gt;This should support queries that which want to use a covered index; then complex intersections would never need to hit the document to return the values.&lt;/p&gt;

&lt;p&gt;A more detailed description of index intersection in a traditional RDMS can be found here:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;http://www.sql-server-performance.com/2003/index-intersection/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://www.sql-server-performance.com/2003/index-intersection/&lt;/a&gt;&lt;/p&gt;</description>
                <environment></environment>
        <key id="16821">SERVER-3071</key>
            <summary>Index Intersection</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="hari.khalsa@10gen.com">hari.khalsa@10gen.com</assignee>
                                    <reporter username="scotthernandez">Scott Hernandez</reporter>
                        <labels>
                            <label>indexing</label>
                            <label>performance</label>
                            <label>query_triage</label>
                    </labels>
                <created>Mon, 9 May 2011 21:55:47 +0000</created>
                <updated>Tue, 12 Jul 2016 00:19:00 +0000</updated>
                            <resolved>Fri, 20 Dec 2013 22:11:17 +0000</resolved>
                                                    <fixVersion>2.5.5</fixVersion>
                                    <component>Index Maintenance</component>
                    <component>Querying</component>
                                        <votes>71</votes>
                                    <watches>77</watches>
                                                                                                                <comments>
                            <comment id="509479" author="xgen-internal-githook" created="Tue, 4 Mar 2014 22:25:37 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;dstorch&apos;, u&apos;name&apos;: u&apos;David Storch&apos;, u&apos;email&apos;: u&apos;david.storch@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3071&quot; title=&quot;Index Intersection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3071&quot;&gt;&lt;del&gt;SERVER-3071&lt;/del&gt;&lt;/a&gt; penalize fetches during ranking&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/33912523e40467e1a41bed60ed7e198200a040ad&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/33912523e40467e1a41bed60ed7e198200a040ad&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="508711" author="nevi_me" created="Tue, 4 Mar 2014 04:15:01 +0000"  >&lt;p&gt;Thanks David, that clarifies a lot of things. For example, if I would have read somewhere whilst following this issue that the limit of 2 indices is because of arrays, I&apos;d have understood. Certainly will be interesting to see how an increase in the indices works in future versions, especially if there&apos;s a good was of applying heuristics to the search space. &lt;/p&gt;

&lt;p&gt;Thanks again, I for one really appreciate it!&lt;/p&gt;</comment>
                            <comment id="508454" author="david.storch" created="Mon, 3 Mar 2014 22:00:46 +0000"  >&lt;p&gt;Hi all,&lt;/p&gt;

&lt;p&gt;Here are answers to some common questions that we are receiving about the index intersection feature, to be included in the upcoming v2.6 release. We will add to this list of Q&amp;amp;A as we hear from users.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Q: What is the maximum number of indices that can be intersected?&lt;/b&gt;&lt;br/&gt;
A: Only intersections between 2 indices are currently considered. Without this limitation, the optimizer may have to consider every possible combination of indices, which could be prohibitively large. (Alternately, the size of the search space could be heuristically pruned, but this will not be part of v2.6.)&lt;/p&gt;

&lt;p&gt;The limitation of 2 index scans does not apply to &quot;self-intersections&quot;. To understand &quot;self-intersections&quot;, consider the common use case of indexing an array of attributes, e.g. attributes: [&quot;electronics&quot;, &quot;homeGoods&quot;, &quot;laptops&quot;]. Say we query for {attributes: {$all: [&quot;electronics&quot;, &quot;laptops&quot;]}}. In previous versions of MongoDB, this would retrieve all documents with the index key &quot;electronics&quot;, and then filter the result set by documents that also contain the tag &quot;laptops&quot;. We cannot simply retrieve all documents with either the index key &quot;electronics&quot; or the index key &quot;laptops&quot; because then the result set could contain documents that contain one of the two tags but not both. With index intersection, we can now retrieve the documents with &quot;electronics&quot; and the documents with &quot;laptops&quot; using two separate index scans, and then fetch only the intersection of the two scans. This &quot;self-intersection&quot; can considerably reduce the number of complete documents that the query execution needs to retrieve. We currently cap the number of &quot;self-intersection&quot; scans at 10.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Q: How does the performance of index intersection compare to compound indices? When should I use a compound index, and when should I rely on index intersection?&lt;/b&gt;&lt;br/&gt;
A: Using a compound index is almost always faster than index intersection, and the query optimizer will not even consider an index intersection plan if a compound index is available over the same fields. Index intersection generally needs to scan more index keys than a compound index in order to find documents satisfying the relevant predicates. As such, index intersection should not be considered a replacement for compound indices. If a large percentage of your workload is to look up documents on fields &apos;a&apos;, &apos;b&apos;, and &apos;c&apos;, then it is probably worth building a compound index on these fields.&lt;/p&gt;

&lt;p&gt;Relying on index intersection may make more sense if you have a more varied workload: sometimes you look up by (a, b), sometimes by (b, c), sometimes by (c, d), sometimes just by &apos;b&apos;, etc. While you certainly could solve this problem by building many compound indices, it may make more sense to build several single-field indices. Rather than predicting which pairs of fields you will query on, you can use index intersection determine these pairs in an &lt;em&gt;ad hoc&lt;/em&gt; fashion.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Q: Will v2.6 support query + sort intersection?&lt;/b&gt;&lt;br/&gt;
A: Yes. One caveat is that the query must contain predicates over both indices.  Consider a collection with indices on {zipcode:1} and {last_name:1}.  We will not perform index intersection for db.phonebook.find({zipcode: &quot;12345&quot;}).sort({last_name: 1}). On the other hand, we will consider an index intersection plan for db.phonebook.find({zipcode: &quot;12345&quot;, last_name: /^S/}).sort({last_name: 1}).&lt;/p&gt;

&lt;p&gt;The reason the sort field is required in the query predicate is because without it we would have to scan the entire index in order to obtain the sort.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Q: When will the query optimizer select index intersection plans over single-index plans?&lt;/b&gt;&lt;br/&gt;
A: The query optimizer may select index intersection plans when the following conditions hold:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;Most of the documents in the relevant collection are disk-resident. The advantage of index intersection is that it can avoid fetching complete documents when the size of the intersection is small. If the documents are already in memory, there is nothing to gain by avoiding fetches.&lt;/li&gt;
	&lt;li&gt;The query predicates are single point intervals, rather than range predicates or a set of intervals. Queries over single point intervals return documents sorted by disk location, which allows the optimizer to select plans that compute the intersection in a non-blocking fashion. This is generally faster than the alternative mode of computing the intersection, which is to build a hash table with the results from one index, and then probe it with the results from the second index.&lt;/li&gt;
	&lt;li&gt;Neither of the indices to be intersected are highly selective. If one of the indices is selective then the optimizer will choose a plan which simply scans this selective index.&lt;/li&gt;
	&lt;li&gt;The size of the intersection is small relative to the number of index keys scanned by either single-index solution. In this case the query executor can look at a smaller set of documents using index intersection, potentially allowing us to reap the benefits of fewer fetches from disk.&lt;/li&gt;
&lt;/ol&gt;
</comment>
                            <comment id="508241" author="jimwang" created="Mon, 3 Mar 2014 19:21:45 +0000"  >&lt;p&gt;We are also looking for query + sort index intersection.&lt;br/&gt;
Currently we have to create lots of compound indexes to support query + sort.&lt;br/&gt;
It seems index intersection only applies to query for the coming 2.6 release.&lt;br/&gt;
Is there any plan to support query + sort in the future release? &lt;/p&gt;</comment>
                            <comment id="503559" author="paulkon" created="Sat, 22 Feb 2014 01:31:40 +0000"  >&lt;p&gt;So,&lt;/p&gt;

&lt;p&gt;a) will this support query + sort index intersection?&lt;br/&gt;
b) what is the max number of intersected indexes per query + sort?&lt;br/&gt;
c) what is the performance difference between intersections and compound indexes?&lt;br/&gt;
d) can this be considered a replacement for compound indexes?&lt;/p&gt;

&lt;p&gt;This would make my index optimization an order of magnitude easier if it works essentially as a replacement for large compound indexes.&lt;/p&gt;</comment>
                            <comment id="481296" author="sallgeud" created="Mon, 13 Jan 2014 22:00:16 +0000"  >&lt;p&gt;Moving this from 1 to 2 indexes used at a time is essentially a non-change from usability.  Those of us looking for intersections were hoping for a much larger set (8,16,32, etc).  &lt;/p&gt;</comment>
                            <comment id="477406" author="mattparlane" created="Sun, 5 Jan 2014 23:33:30 +0000"  >&lt;p&gt;Quick question about this: will this feature allow me to simply have one index on every field I might ever need to filter on (obviously up to the 64 index limit), or is that too simplistic?&lt;/p&gt;</comment>
                            <comment id="472841" author="sallgeud" created="Wed, 18 Dec 2013 22:45:33 +0000"  >&lt;p&gt;Excellent to see this going into the next major release.  We&apos;ll be watching this and testing when it&apos;s out.  Any chance this will work with filter &amp;amp; sorting at the same time?&lt;/p&gt;</comment>
                            <comment id="450749" author="tobias74" created="Mon, 4 Nov 2013 10:16:02 +0000"  >&lt;p&gt;Is &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-4572&quot; title=&quot;Geospatial index cannot be used in $and criteria of a query?&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-4572&quot;&gt;&lt;del&gt;SERVER-4572&lt;/del&gt;&lt;/a&gt; really depended on by this issue? Making use of the correct geospatial index is possible with a workaround in the query formulation: Using  a comma to express a logical conjunction works fine, while using &apos;$and&apos; in the query will cause the bug as described in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-4572&quot; title=&quot;Geospatial index cannot be used in $and criteria of a query?&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-4572&quot;&gt;&lt;del&gt;SERVER-4572&lt;/del&gt;&lt;/a&gt;. &lt;/p&gt;

&lt;p&gt;Do &apos;,&apos; and &apos;$and&apos; make a difference when it comes to choosing the right index?&lt;/p&gt;</comment>
                            <comment id="283780" author="tad" created="Thu, 7 Mar 2013 22:15:39 +0000"  >&lt;p&gt;2.5.w is just a scheduling mechanism to pull something to the front of the 2.5.x group.  There are enough tickets marked for 2.5.x (meaning planned for some release within the 2.5/2.6 version cycle) that it is helpful to use this method to flag a subset of them.  &quot;w&quot; comes before &quot;x&quot; in the alphabet, so it&apos;s just a sorting tool.&lt;/p&gt;</comment>
                            <comment id="283701" author="rgpublic" created="Thu, 7 Mar 2013 20:59:50 +0000"  >&lt;p&gt;Sorry for the spam but what exactly is the difference between 2.5.x and 2.5.w? I&apos;ve searched all over the MongoDB and JIRA site and cannot find any information on this peculiar versioning scheme.&lt;/p&gt;</comment>
                            <comment id="183265" author="nick.gerner@gmail.com" created="Tue, 6 Nov 2012 21:00:57 +0000"  >&lt;p&gt;any thoughts on if this is anywhere in the list of priorities?&lt;/p&gt;</comment>
                            <comment id="71867" author="jeff.yemin" created="Fri, 9 Dec 2011 02:58:59 +0000"  >&lt;p&gt;That link is dead.  I think this is the new location: &lt;a href=&quot;http://www.sql-server-performance.com/2003/index-intersection/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://www.sql-server-performance.com/2003/index-intersection/&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="64310" author="ecowden" created="Wed, 2 Nov 2011 19:21:37 +0000"  >&lt;p&gt;This would be an absolutely huge feature for us.&lt;/p&gt;

&lt;p&gt;When we present one of our main collections to users, we would like to allow users to choose to search by one field and then sort by another.  It would be wonderful if we could index each of the sort fields independently of the original search fields.&lt;/p&gt;</comment>
                            <comment id="34029" author="scotthernandez" created="Mon, 23 May 2011 22:26:18 +0000"  >&lt;p&gt;It would be nice if this supported all the features listed here: &lt;a href=&quot;http://sql-server-2000.net/MS.Press-Microsoft.SQL.Server4/_index_intersection_236.htm&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://sql-server-2000.net/MS.Press-Microsoft.SQL.Server4/_index_intersection_236.htm&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This would support a query for two (or more) different fields based on single value indexes.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="19410">SERVER-3414</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="27368">SERVER-4572</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="57154">SERVER-7742</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="66611">SERVER-8753</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="17261">SERVER-3118</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="66705">SERVER-8768</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="61603">SERVER-8140</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="11761">SERVER-1000</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="69959">SERVER-9148</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="66858">SERVER-8790</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>15.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 2 Nov 2011 19:21:37 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        9 years, 50 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-3414'>SERVER-3414</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            9 years, 50 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>sallgeud</customfieldvalue>
            <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
            <customfieldvalue>ecowden</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>hari.khalsa@10gen.com</customfieldvalue>
            <customfieldvalue>jeff.yemin@mongodb.com</customfieldvalue>
            <customfieldvalue>jimwang</customfieldvalue>
            <customfieldvalue>mattparlane</customfieldvalue>
            <customfieldvalue>nevi_me</customfieldvalue>
            <customfieldvalue>nick.gerner@gmail.com</customfieldvalue>
            <customfieldvalue>paulkon</customfieldvalue>
            <customfieldvalue>rgpublic</customfieldvalue>
            <customfieldvalue>scotthernandez</customfieldvalue>
            <customfieldvalue>tad</customfieldvalue>
            <customfieldvalue>tobias74</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrp03r:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrfx9b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5728</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrpjdb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>