<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:32:53 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-13780] sparsePolicy for sparse compound indexes</title>
                <link>https://jira.mongodb.org/browse/SERVER-13780</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;This patch was initially posted as a comment to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-2193&quot; title=&quot;Sparse indexes only support a single field&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-2193&quot;&gt;&lt;del&gt;SERVER-2193&lt;/del&gt;&lt;/a&gt; but that ticket is on a bit of a different topic, so we discussed with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=thomasr&quot; class=&quot;user-hover&quot; rel=&quot;thomasr&quot;&gt;thomasr&lt;/a&gt; that it should rather be made an independent ticket. Original comment follows:&lt;/p&gt;

&lt;p&gt;Support for sparse index with multiple fields is, as far as I can see, implemented in the current code base and has been for a long time (and with written tests). There is a uassert in the code that suppose to prevent a user from creating a sparse index with multiple fields but the implementation of this is wrong so it will never kick in.&lt;/p&gt;

&lt;p&gt;So, if a user creates a sparse index with multiple fields it will work. The semantics for the current implementation is; &quot;exclude a document from the index if &lt;em&gt;all&lt;/em&gt; index fields are &lt;em&gt;missing&lt;/em&gt; from the document&quot;.&lt;/p&gt;

&lt;p&gt;This &quot;mode&quot; of the index might benefit some, but according to many of the wishes in the discussion in this issue and in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-785&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-785&lt;/a&gt; the semantics folks are looking for is; &quot;only include a document in the index if &lt;em&gt;all&lt;/em&gt; index fields are &lt;em&gt;present&lt;/em&gt; in the document&quot;.&lt;/p&gt;

&lt;p&gt;Getting support for this second &quot;mode&quot; of the index is simply a matter of changing &lt;tt&gt;numNotFound == _spec._nFields&lt;/tt&gt; to &lt;tt&gt;numNotFound != 0&lt;/tt&gt; here: &lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/db/indexkey.cpp#L429&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/master/src/mongo/db/indexkey.cpp#L429&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Provided that I have not missed any complicated corner case regarding this, I have the following suggestions:&lt;/p&gt;

&lt;p&gt;1. Change the documentation so that it is clear that sparse index with multiple fields is supported.&lt;br/&gt;
2. Add a additional config parameter that can be used together with the &lt;tt&gt;sparse: true&lt;/tt&gt; option to flip the behavior of the index according to the second semantics above.&lt;/p&gt;

&lt;p&gt;You can find the code/patch that does this (with test case) here: &lt;a href=&quot;https://github.com/johanhedin/mongo/commits/SERVER-2193&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/johanhedin/mongo/commits/SERVER-2193&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;With this patch you could create a index like this:&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;db.collection.ensureIndex({ a: 1, b: 1 }, { sparse: &lt;/span&gt;&lt;span style=&quot;color: #006699; font-weight: bold; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;true&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;, sparsePolicy: &lt;/span&gt;&lt;span style=&quot;color: blue; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;&quot;include&quot;&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; })&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;and only documents where &lt;em&gt;both&lt;/em&gt; &lt;tt&gt;a&lt;/tt&gt; &lt;em&gt;and&lt;/em&gt; &lt;tt&gt;b&lt;/tt&gt; are present will be included in the index. If &lt;tt&gt;sparsePolicy&lt;/tt&gt; is left out (the default) the index will work as before. And of course, the name &lt;tt&gt;sparsePolicy&lt;/tt&gt; is just an suggestion.&lt;/p&gt;

&lt;p&gt;I have only addressed v1 indexes but that same seem to be doable for v0 indexes as well if that is desired.&lt;/p&gt;

&lt;p&gt;I&apos;m happy to create a pull request if this is something you would consider. For my use case, this would be a HUGE improvement since we are starting to scale from hundreds of millions of documents to hundreds of billions of documents and RAM usage for our indexes is a big issue costing a lot of money for hardware that just store &quot;empty&quot; values.&lt;/p&gt;</description>
                <environment></environment>
        <key id="133987">SERVER-13780</key>
            <summary>sparsePolicy for sparse compound indexes</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="hari.khalsa@10gen.com">hari.khalsa@10gen.com</assignee>
                                    <reporter username="henrik.ingo@mongodb.com">Henrik Ingo</reporter>
                        <labels>
                    </labels>
                <created>Tue, 29 Apr 2014 18:54:05 +0000</created>
                <updated>Fri, 17 Apr 2015 12:10:10 +0000</updated>
                            <resolved>Tue, 5 Aug 2014 08:15:02 +0000</resolved>
                                                                    <component>Index Maintenance</component>
                                        <votes>5</votes>
                                    <watches>12</watches>
                                                                                                                <comments>
                            <comment id="681460" author="henrik.ingo@10gen.com" created="Tue, 5 Aug 2014 08:15:02 +0000"  >&lt;p&gt;SERVER-785 is a superset of this patch, and has now been scheduled into Planning Bucket A.&lt;/p&gt;</comment>
                            <comment id="567325" author="henrik.ingo@10gen.com" created="Tue, 29 Apr 2014 20:32:48 +0000"  >&lt;p&gt;To followup on Thomas comments Re: SERVER-785. To review &quot;what do people really want&quot;, I sampled a few emails from mongodb-user and mongodb-dev. Some people of course really have ideas where arbitrary filtered index is needed. But it seems many people could actually benefit from the proposed {&lt;tt&gt;sparsePolicy : &quot;allFields&quot;&lt;/tt&gt;}.&lt;/p&gt;

&lt;p&gt;For example:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://groups.google.com/forum/#!search/mongodb$20exclude$20fields$20sparse$20index/mongodb-user/my481lz38x4/k6ZS6Qc98CoJ&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;DaveC&lt;/a&gt; is proposing a solution based on &quot;allFields&quot; semantics. (Asya then corrects him that what he proposes isn&apos;t currently possible.)&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://groups.google.com/forum/#!search/mongodb$20exclude$20fields$20sparse$20index/mongodb-user/xIIiU0A3irE/BYpeeiFnCY4J&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Albert Kam&lt;/a&gt; suggests a usage that as such would require filtering by value, but actually what he wants could be achieved with simple &quot;allFields&quot; semantics too.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://groups.google.com/forum/#!search/mongodb$20exclude$20fields$20sparse$20index/mongodb-user/rinmD_wmYxQ/UNEAKFh-8_8J&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Clemo&lt;/a&gt; seems to ask for &quot;allFields&quot; semantics too.&lt;/p&gt;

&lt;p&gt;(So in this sample of 3, a 100% actually would be well served by this patch.)&lt;/p&gt;</comment>
                            <comment id="567215" author="henrik.ingo@10gen.com" created="Tue, 29 Apr 2014 18:56:13 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-13780&quot; title=&quot;sparsePolicy for sparse compound indexes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-13780&quot;&gt;&lt;del&gt;SERVER-13780&lt;/del&gt;&lt;/a&gt; was created from a comment in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-2193&quot; title=&quot;Sparse indexes only support a single field&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-2193&quot;&gt;&lt;del&gt;SERVER-2193&lt;/del&gt;&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="84364">SERVER-10403</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="13901">SERVER-2193</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 29 Apr 2014 19:24:35 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        9 years, 28 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>marc.schwering</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            9 years, 28 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>hari.khalsa@10gen.com</customfieldvalue>
            <customfieldvalue>henrik.ingo@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlvv3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hryg0f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>115161</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsgy1j:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>