<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:27:02 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-74293] Improve spilling algorithm for HashLookupStage</title>
                <link>https://jira.mongodb.org/browse/SERVER-74293</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;It looks to me like &lt;tt&gt;HashLookupStage&lt;/tt&gt; suffers from a similar performance problem as was corrected for &lt;tt&gt;HashAggStage&lt;/tt&gt; in related ticket &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-70395&quot; title=&quot;Slot-Based Engine too aggressively uses disk for $group and is slow&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-70395&quot;&gt;&lt;del&gt;SERVER-70395&lt;/del&gt;&lt;/a&gt;. That is, the data that is spilled either to &lt;tt&gt;HashLookupStage::_recordStoreHt&lt;/tt&gt; and &lt;tt&gt;HashLookupStage::_recordStoreBuf&lt;/tt&gt; may be repeatedly deserialized when consuming the input from the outer child.&lt;/p&gt;

&lt;p&gt;My thinking after discussing with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=anna.wawrzyniak%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;anna.wawrzyniak@mongodb.com&quot;&gt;anna.wawrzyniak@mongodb.com&lt;/a&gt; is that we should probably choose to do nothing about this problem right now, and instead just close this ticket as &quot;Won&apos;t Do&quot;. I&apos;m filing the ticket so that we have this conversation and make an explicit decision about it as a team. The primary reason to avoid scheduling this improvement is that we currently only use &lt;tt&gt;HashLookupStage&lt;/tt&gt; to support the pushdown of $lookup to SBE. There are heuristics in place so that we only choose a hash-based plan for $lookup when the data size is sufficiently small. This makes spilling possible but extremely unlikely. For this reason, we probably don&apos;t want to invest lots of engineering effort into spilling performance for hash-based $lookup plans in SBE right now. Furthermore, the regular &lt;tt&gt;HashJoinStage&lt;/tt&gt; in SBE does not support spilling yet. It would be wise to design and implement a good spilling algorithm for regular hash join before we try to implement something similar for the special case of &lt;tt&gt;HashLookupStage&lt;/tt&gt;.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2272637">SERVER-74293</key>
            <summary>Improve spilling algorithm for HashLookupStage</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="1" iconUrl="https://jira.mongodb.org/images/icons/statuses/open.png" description="">Open</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-query-execution">Backlog - Query Execution</assignee>
                                    <reporter username="david.storch@mongodb.com">David Storch</reporter>
                        <labels>
                    </labels>
                <created>Wed, 22 Feb 2023 20:56:24 +0000</created>
                <updated>Tue, 14 Mar 2023 17:36:16 +0000</updated>
                                                                            <component>Query Execution</component>
                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="5236970" author="david.storch" created="Tue, 28 Feb 2023 16:13:23 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=mihai.andrei%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;mihai.andrei@mongodb.com&quot;&gt;mihai.andrei@mongodb.com&lt;/a&gt; I&apos;m sticking this in m4 of the SBE perf project so that we can review it as a team. I think we should probably close it as &quot;Won&apos;t Do&quot; as I suggested above, but I wanted to run it by the SBE perf team first.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="2155277">SERVER-70395</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25125"><![CDATA[Query Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        49 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-3243</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>mihai.andrei@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            49 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-query-execution</customfieldvalue>
            <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1xecf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i1ftfs:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10555" key="com.atlassian.jira.plugin.system.customfieldtypes:float">
                        <customfieldname>Story Points</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1x0hr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>