<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:39:46 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-16042] Optimise $all/$and to select smallest subset as initial index bounds</title>
                <link>https://jira.mongodb.org/browse/SERVER-16042</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Currently clauses in a $all will be evaluated in the order that they are provided to the query. This can lead to cases where a superset is being searched to find values that are contained within a smaller subset of the wanted values.&lt;/p&gt;

&lt;p&gt;Consider trying to find all documents that have tags showing that they are an Italian and have a Michelin star within a collection of restaurants. There will be a large number of documents with the tag &quot;Italian&quot; but very few with the tag &quot;Michelin&quot;. &lt;/p&gt;

&lt;p&gt;If the tags are entered into the $all clause in alphabetical order, the search will take longer as the larger set of &quot;Italian&quot; documents must be searched again for the &quot;Michelin&quot; tag. If the tag order is reversed, then we search the smaller set of &quot;Michelin&quot; matching documents to find those who match &quot;Italian&quot;.&lt;/p&gt;

&lt;p&gt;The optimisation here would be to remove the dependency on the order of documents in an $all clause.&lt;/p&gt;

&lt;p&gt;Changes in this case should handle situations where the entities within the $all clause are not simply fields, but also documents which may have several levels of depth.&lt;/p&gt;</description>
                <environment></environment>
        <key id="168131">SERVER-16042</key>
            <summary>Optimise $all/$and to select smallest subset as initial index bounds</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="david.storch@mongodb.com">David Storch</assignee>
                                    <reporter username="david.hows">David Hows</reporter>
                        <labels>
                    </labels>
                <created>Mon, 10 Nov 2014 03:12:51 +0000</created>
                <updated>Wed, 7 Sep 2016 01:58:12 +0000</updated>
                            <resolved>Thu, 1 Oct 2015 22:29:27 +0000</resolved>
                                                    <fixVersion>2.6.12</fixVersion>
                    <fixVersion>3.0.7</fixVersion>
                    <fixVersion>3.1.9</fixVersion>
                                    <component>Querying</component>
                                        <votes>4</votes>
                                    <watches>15</watches>
                                                                                                                <comments>
                            <comment id="1091722" author="xgen-internal-githook" created="Wed, 18 Nov 2015 15:19:01 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;dstorch&apos;, u&apos;name&apos;: u&apos;David Storch&apos;, u&apos;email&apos;: u&apos;david.storch@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-16042&quot; title=&quot;Optimise $all/$and to select smallest subset as initial index bounds&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-16042&quot;&gt;&lt;del&gt;SERVER-16042&lt;/del&gt;&lt;/a&gt; generate a candidate plan for each predicate over the leading field of a multikey index&lt;br/&gt;
Branch: v2.6&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/461c5d496e9e1896be4371478867117ec552d7a5&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/461c5d496e9e1896be4371478867117ec552d7a5&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1049387" author="xgen-internal-githook" created="Thu, 1 Oct 2015 23:59:12 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;dstorch&apos;, u&apos;name&apos;: u&apos;David Storch&apos;, u&apos;email&apos;: u&apos;david.storch@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-16042&quot; title=&quot;Optimise $all/$and to select smallest subset as initial index bounds&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-16042&quot;&gt;&lt;del&gt;SERVER-16042&lt;/del&gt;&lt;/a&gt; fix lint&lt;br/&gt;
Branch: v3.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/ea38cdc9c2aa753d7720913fe92d1763bfb1763e&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/ea38cdc9c2aa753d7720913fe92d1763bfb1763e&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1049352" author="xgen-internal-githook" created="Thu, 1 Oct 2015 23:03:38 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;dstorch&apos;, u&apos;name&apos;: u&apos;David Storch&apos;, u&apos;email&apos;: u&apos;david.storch@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-16042&quot; title=&quot;Optimise $all/$and to select smallest subset as initial index bounds&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-16042&quot;&gt;&lt;del&gt;SERVER-16042&lt;/del&gt;&lt;/a&gt; generate a candidate plan for each predicate over the leading field of a multikey index&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 4372e7b826bbaff94d995c0a37d9a8bf29378e48)&lt;/p&gt;

&lt;p&gt;Conflicts:&lt;br/&gt;
	src/mongo/db/query/query_planner_array_test.cpp&lt;br/&gt;
	src/mongo/db/query/query_planner_geo_test.cpp&lt;br/&gt;
	src/mongo/db/query/query_planner_test.cpp&lt;br/&gt;
Branch: v3.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/e4964e0b77ee3d055946fb88bf37338a6a1906f1&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/e4964e0b77ee3d055946fb88bf37338a6a1906f1&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1049307" author="xgen-internal-githook" created="Thu, 1 Oct 2015 22:28:46 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;dstorch&apos;, u&apos;name&apos;: u&apos;David Storch&apos;, u&apos;email&apos;: u&apos;david.storch@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-16042&quot; title=&quot;Optimise $all/$and to select smallest subset as initial index bounds&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-16042&quot;&gt;&lt;del&gt;SERVER-16042&lt;/del&gt;&lt;/a&gt; generate a candidate plan for each predicate over the leading field of a multikey index&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/4372e7b826bbaff94d995c0a37d9a8bf29378e48&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/4372e7b826bbaff94d995c0a37d9a8bf29378e48&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="813438" author="david.storch" created="Mon, 26 Jan 2015 19:36:16 +0000"  >&lt;p&gt;One more note: the query planner rewrites an $all to an $and. Everything described here for $all also applies to an $and.&lt;/p&gt;</comment>
                            <comment id="764559" author="david.storch" created="Fri, 14 Nov 2014 17:35:09 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=petrbela&quot; class=&quot;user-hover&quot; rel=&quot;petrbela&quot;&gt;petrbela&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Great question. We closed &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-1000&quot; title=&quot;$all with query optimizer&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-1000&quot;&gt;&lt;del&gt;SERVER-1000&lt;/del&gt;&lt;/a&gt; at a time when the thinking was that index intersection would fully resolve this issue. However, we have since realized that index intersection is not sufficient. This ticket tracks our work on a more comprehensive fix, so please continue to watch for updates.&lt;/p&gt;

&lt;p&gt;Here&apos;s why index intersection is not necessarily the answer for your &quot;halloween&quot; and &quot;intercontinental&quot; example. Currently, the query planner will consider two plans using the index on &lt;tt&gt;tags&lt;/tt&gt;:&lt;/p&gt;

&lt;ol&gt;
	&lt;li&gt;Fetch all 4 million documents with the &quot;halloween&quot; tag, then filter by &quot;intercontinental&quot;.&lt;/li&gt;
	&lt;li&gt;Sort-based index intersection of the documents with &quot;halloween&quot; and the documents with &quot;intercontinental&quot;.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;The sort-based index intersection plan requires scanning 4 million &quot;halloween&quot; keys plus 4 thousand &quot;intercontinental&quot; keys, and also incurs some CPU overhead as it computes the intersection between these sets of keys. Although this is possibly cheaper than the plan that only scans &quot;halloween&quot; keys, it still can do a lot of unnecessary scanning.&lt;/p&gt;

&lt;p&gt;The planner currently fails to generate and rank a plan which fetches the 4 thousand documents with the &quot;intercontinental&quot; tag and then filters by &quot;halloween&quot;. This is probably the best plan, but it is never even considered. One of a few fixes that have been discussed by our dev team is to simply generate and rank additional plans, in this case ranking the plan that scans &quot;intercontinental&quot; keys in addition to the one that scans &quot;halloween&quot; keys.&lt;/p&gt;

&lt;p&gt;Best,&lt;br/&gt;
Dave&lt;/p&gt;</comment>
                            <comment id="764181" author="petrbela" created="Fri, 14 Nov 2014 05:35:11 +0000"  >&lt;p&gt;We&apos;ve just run into the same issue.&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;db.assets.find({ tags: { $all: [&apos;halloween&apos;, &apos;intercontinental&apos;] } })&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;fetches all &apos;halloween&apos; documents (about 4 million in our case) and then scans them for &apos;intercontinental&apos; (around 4000).&lt;/p&gt;

&lt;p&gt;I was wondering if the index intersection was supposed to fix that? This comment &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-1000?focusedCommentId=473980&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-473980&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-1000?focusedCommentId=473980&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-473980&lt;/a&gt; seems to imply that it has been resolved, but as of 2.6.5, it still behaves as described above. Or is index intersection not yet (ever?) supported for array fields?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="226052">SERVER-19999</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="234267">SERVER-20913</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="22461">SERVER-3892</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="107879">SERVER-12499</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="142251">SERVER-14264</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="105000">SERVER-12281</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="314479">DOCS-8779</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="202724">SERVER-18364</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="11761">SERVER-1000</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12451" key="com.atlassian.jira.plugin.system.customfieldtypes:multiversion">
                        <customfieldname>Backport Completed</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="15782">2.6.12</customfieldvalue>
    <customfieldvalue id="15689">3.0.7</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 10 Nov 2014 06:49:58 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        8 years, 13 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>andrew.ryder@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            8 years, 13 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>david.hows</customfieldvalue>
            <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>petrbela</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlji7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hreutj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>147012</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="490">Quint Iteration 3</customfieldvalue>
    <customfieldvalue id="705">QuInt A (10/12/15)</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11861" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>User Summary</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="11857"><![CDATA[Requested]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrtg8f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>