<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:30:32 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-32540] Make partial index subset analysis consider $elemMatch object a subset of $exists</title>
                <link>https://jira.mongodb.org/browse/SERVER-32540</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;The query explainer don&apos;t consider the keys in $elemMatch expression when to decide index boundaries. Instead, it scans all entries in the selected index.&lt;/p&gt;

</description>
                <environment>mongod --version&lt;br/&gt;
db version v3.4.7&lt;br/&gt;
git version: cf38c1b8a0a8dca4a11737581beafef4fe120bcd&lt;br/&gt;
OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013&lt;br/&gt;
allocator: tcmalloc&lt;br/&gt;
modules: none&lt;br/&gt;
build environment:&lt;br/&gt;
&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;distmod: rhel70&lt;br/&gt;
&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;distarch: x86_64&lt;br/&gt;
&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;target_arch: x86_64&lt;br/&gt;
&lt;br/&gt;
cat /etc/redhat-release &lt;br/&gt;
CentOS Linux release 7.0.1406 (Core)&lt;br/&gt;
</environment>
        <key id="478681">SERVER-32540</key>
            <summary>Make partial index subset analysis consider $elemMatch object a subset of $exists</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-query-optimization">Backlog - Query Optimization</assignee>
                                    <reporter username="huyingming">Alex Hu</reporter>
                        <labels>
                    </labels>
                <created>Thu, 4 Jan 2018 08:25:37 +0000</created>
                <updated>Tue, 6 Dec 2022 03:42:35 +0000</updated>
                                                                            <component>Querying</component>
                                        <votes>3</votes>
                                    <watches>16</watches>
                                                                                                                <comments>
                            <comment id="1813343" author="david.storch" created="Thu, 22 Feb 2018 19:45:37 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=huyingming&quot; class=&quot;user-hover&quot; rel=&quot;huyingming&quot;&gt;huyingming&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thanks for this issue report. The root cause of the behavior you&apos;re seeing is that the planner does not consider the query predicate eligible to use the partial index. I can reproduce this behavior as described, but only when the index has a &lt;tt&gt;partialFilterExpression&lt;/tt&gt;.&lt;/p&gt;

&lt;p&gt;To provide some background: early in the query planning process, the planner undergoes an index selection phase in which it identifies indexes that are relevant to the query. For partial indexes, this involves proving, without running the query, that the predicate is guaranteed to match a subset of the documents which match the &lt;tt&gt;partialFilterExpression&lt;/tt&gt;. This is necessary for correctness, since the partial index must have keys for all matching documents in order for the query plan to be correct. The necessary subset analysis is implemented by &lt;a href=&quot;https://github.com/mongodb/mongo/blob/9c65faa1ea24b92d86f846abd38909b4ef2fe5bc/src/mongo/db/matcher/expression_algo.h#L72&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;expression_algo::isSubsetOf&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;tt&gt;isSubsetOf&lt;/tt&gt; behaves conservatively: if it cannot prove that a subset relationship exists, it returns &lt;tt&gt;false&lt;/tt&gt;, and the partial index will not be considered relevant to the query. This can happen either because 1) the code to prove the subset relationship for this particular predicate type has not been implemented, or 2) it is impossible to prove that the subset relationship exists without inspecting the data. You have stumbled upon an instance of the former scenario. I&apos;m fairly certain that&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;var query = {a: {$elemMatch: {b: 3}};&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;is guaranteed to match a subset of the documents matching&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;var partialFilterExpression = {&quot;a.b&quot;: {$exists: true}};&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;However, this optimization has not yet been implemented. This is expected, since not all known optimizations of this variety have been implemented (see, e.g., &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-17853&quot; title=&quot;Allow more complex expressions to be specified in partial index document filter&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-17853&quot;&gt;SERVER-17853&lt;/a&gt;). I will leave this ticket open as an improvement request, and retitle it to &quot;Make partial index subset analysis consider $elemMatch object a subset of $exists&quot;. I am also going to direct this ticket to the Query Team for triage. It seems likely that we will pursue this work item only when we are making other partial index planning improvements, such as &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-17853&quot; title=&quot;Allow more complex expressions to be specified in partial index document filter&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-17853&quot;&gt;SERVER-17853&lt;/a&gt; or &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18884&quot; title=&quot;Allow $in to be specified in partial index document filter&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-18884&quot;&gt;&lt;del&gt;SERVER-18884&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;One more thing: the [MinKey, MaxKey] index bounds you mention in your original report of this issue are a consequence of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-26413&quot; title=&quot;Hinting an incompatible partial index can return incomplete results&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-26413&quot;&gt;SERVER-26413&lt;/a&gt;. The system currently allows a user to hint a partial index that is not eligible to answer the query. In general, this can cause the planner to produce incorrect plans. In your case, the plan happens to be correct, but is inefficient.&lt;/p&gt;

&lt;p&gt;Best,&lt;br/&gt;
Dave&lt;/p&gt;</comment>
                            <comment id="1769843" author="huyingming" created="Wed, 10 Jan 2018 03:22:49 +0000"  >&lt;p&gt;One comment: This issue is regardless with the partial filter from the index spec. That means the issue is the same even if remove the partialFilterExpression from the index spec.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="341205">SERVER-27522</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="320135">SERVER-26413</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="193542">SERVER-17853</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25126"><![CDATA[Query Optimization]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[5002K00000uy8P8QAI]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 9 Jan 2018 14:12:05 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 50 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-283</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 50 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>huyingming</customfieldvalue>
            <customfieldvalue>backlog-query-optimization</customfieldvalue>
            <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htnbc7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr2bov:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                    <customfieldvalue><![CDATA[kelsey.schubert@mongodb.com]]></customfieldvalue>
    

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htmxgn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>