<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:57:54 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-63482] Clean up filter checking in partial indexes</title>
                <link>https://jira.mongodb.org/browse/SERVER-63482</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;During the review of the &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-63251&quot; title=&quot;Index API refactoring to enable column store indexes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-63251&quot;&gt;&lt;del&gt;SERVER-63251&lt;/del&gt;&lt;/a&gt; changes, a few issues with how we check the filters for filtered indexes were noticed that should be cleaned up.&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;The filter checking logic occurs in both &lt;tt&gt;IndexCatalogImpl&lt;/tt&gt; and &lt;tt&gt;SortedDataIndexAccessMethod&lt;/tt&gt;, but it should probably be the responsibility of only one of the two
	&lt;ul&gt;
		&lt;li&gt;In particular ICI handles the insert path while SDIAM handles updates. (Removes are unfiltered, I think to ensure we clean up stale index keys if we modify the Matcher so that a document no longer matches)&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;There is an extra filter check at the &lt;tt&gt;indexKeys&lt;/tt&gt;/&lt;tt&gt;unindexKeys&lt;/tt&gt; layer (renamed to &lt;tt&gt;(un)indexKeysOrWriteToSideTable&lt;/tt&gt; in that patch) to handle a hybrid index building edge case
	&lt;ul&gt;
		&lt;li&gt;Both of these have comments that say they are deferring to the &lt;tt&gt;IndexAccessMethod&lt;/tt&gt; for filtering in the non-hybrid case, but there doesn&apos;t seem to be any check on that path.&lt;/li&gt;
		&lt;li&gt;I think the check in &lt;tt&gt;indexKeys&lt;/tt&gt; is fully redundant with the checks on the insert and update paths, so it can just be removed.&lt;/li&gt;
		&lt;li&gt;The check in &lt;tt&gt;unindexKeys&lt;/tt&gt; is redundant on updates, but probably needed for removes. I think the best option would be to remove the check from &lt;tt&gt;unindexKeys&lt;/tt&gt; and conditionally check the filter on the remove path only if we are hybrid building (in which case we know there are no stale records to remove from a different version of the matcher.&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;If we aren&apos;t checking the filter on steady-state removes, should we also not check the filter for the old document in updates?
	&lt;ul&gt;
		&lt;li&gt;I think updates have the same issue that cause us not to filter on removes. You could even imagine an update that changes which keys get generated, but becomes a no-op because neither the old or new doc matches the filter, then when the document is removed, we remove the wrong keys from the index.&lt;/li&gt;
		&lt;li&gt;Alternatively, with the Stable API work, maybe we don&apos;t need to bypass the filter on removes since we should be able to rely on the matcher semantics not changing. Also, now that MMAPv1 is gone, leaving stale index entries in the rare case where we changed the Matcher semantics is much less harmful. New storage engines don&apos;t reuse recordids (except for clustered collections), and they are required to be able to detect when a RecordId refers to a deleted record rather than just accessing bad memory (RecordIds in mmapv1 were basically just pointers).&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/db/index/index_access_method.cpp#L940-L946&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;This comment&lt;/a&gt; is confusing and/or misleading
	&lt;ul&gt;
		&lt;li&gt;&quot;If the document applies to the filter (which means that it should have never been indexed), do not suppress the error.&quot;&lt;/li&gt;
		&lt;li&gt;What does it mean for a document to &quot;apply&quot; to a filter? I my first guess would be that it matches, but the parenthetical implies that it means that it doesn&apos;t match because we would have index it if it did.&lt;/li&gt;
		&lt;li&gt;The code suppresses the error if it &lt;b&gt;does not&lt;/b&gt; match the filter or if there is no filter. I can&apos;t tell if that is correct or not. It seems suspicious that it doesn&apos;t follow the normal pattern of no filter being treated the same as a filter that matches every document.&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
&lt;/ul&gt;
</description>
                <environment></environment>
        <key id="1980332">SERVER-63482</key>
            <summary>Clean up filter checking in partial indexes</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-server-execution">Backlog - Storage Execution Team</assignee>
                                    <reporter username="mathias@mongodb.com">Mathias Stearn</reporter>
                        <labels>
                            <label>techdebt</label>
                    </labels>
                <created>Wed, 9 Feb 2022 16:19:35 +0000</created>
                <updated>Fri, 20 Jan 2023 19:43:56 +0000</updated>
                                                                                                <votes>0</votes>
                                    <watches>8</watches>
                                                                                                                <comments>
                            <comment id="4374685" author="connie.chen" created="Thu, 24 Feb 2022 16:38:29 +0000"  >&lt;p&gt;Explore where we are inconsistent and come up with a proposal to refactor as part of the investigation. Once we have a proposal we can put this back into triage.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25136"><![CDATA[Storage Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 24 Feb 2022 16:38:29 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 49 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>henrik.edin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 49 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-execution</customfieldvalue>
            <customfieldvalue>connie.chen@mongodb.com</customfieldvalue>
            <customfieldvalue>mathias@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0jo13:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i0yw3c:k</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0ja6f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>