<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:59:16 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-64021] Lookup stage poor performance</title>
                <link>https://jira.mongodb.org/browse/SERVER-64021</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We switch from a SQL database to MongoDb for our latest cloud application and the performance benefits we received were spectacular. We also followed the MongoDb guidelines and used document embedding when possible. Currently none of our standard CRUD operations require any form of join and are all getting executed incredibly fast (our data sets range from 100K to 2M documents).&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Howerver, in order for us to generate certain reports we need to perform certain joins between collections. The MongoDb C# driver that we are using translates the LINQ join query to a lookup stage. The lookup stage performs a search to the joined collection for every entry of the initial collection. We knew that the &quot;join&quot; performance was lacking on MongoDb, and should be avoided, but requiring 20 seconds to join a collection with 200K documents with a collection with 100K documents was totally unexpected.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;This is a known issue and is acknowledged by the MongoDb team &lt;b&gt;7 years ago&lt;/b&gt; and &lt;b&gt;2 solutions&lt;/b&gt; were proposed.&lt;/p&gt;

&lt;p&gt;[&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21312&quot; title=&quot;$lookup should batch query requests&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21312&quot;&gt;SERVER-21312&lt;/a&gt;|&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21312&quot; title=&quot;$lookup should batch query requests&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21312&quot;&gt;SERVER-21312&lt;/a&gt; $lookup should batch query requests - MongoDB Jira]&lt;/p&gt;

&lt;p&gt;[&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21284&quot; title=&quot;$lookup should cache query results&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21284&quot;&gt;SERVER-21284&lt;/a&gt;|&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21284&quot; title=&quot;$lookup should cache query results&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21284&quot;&gt;SERVER-21284&lt;/a&gt; $lookup should cache query results - MongoDB Jira]&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;We would like to know if you are planning to resolve this issue or if this issue is not possible to be resolved (in order for us to try to resolve it using external methods). Granted that this issue has such a &lt;b&gt;devastating performance impact&lt;/b&gt; and has not been resolved for almost 7 years we are lead to believe that it&apos;s the later case.&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Thanks in advance!&lt;/p&gt;</description>
                <environment>Windows 10&lt;br/&gt;
MongoDb C# Driver v. 2.14.1</environment>
        <key id="1990837">SERVER-64021</key>
            <summary>Lookup stage poor performance</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="edwin.zhou@mongodb.com">Edwin Zhou</assignee>
                                    <reporter username="paplabros@gmail.com">Labros Papadopoulos</reporter>
                        <labels>
                            <label>Aggregation</label>
                            <label>Lookup</label>
                            <label>Performance</label>
                    </labels>
                <created>Sat, 26 Feb 2022 09:16:51 +0000</created>
                <updated>Thu, 17 Mar 2022 17:25:10 +0000</updated>
                            <resolved>Thu, 17 Mar 2022 17:25:03 +0000</resolved>
                                                                    <component>Aggregation Framework</component>
                    <component>Performance</component>
                                        <votes>5</votes>
                                    <watches>10</watches>
                                                                                                                <comments>
                            <comment id="4418799" author="JIRAUSER1257066" created="Thu, 17 Mar 2022 17:25:03 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=paplabros%40gmail.com&quot; class=&quot;user-hover&quot; rel=&quot;paplabros@gmail.com&quot;&gt;paplabros@gmail.com&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thank you for your report. I understand that the performance impact of $lookup can be devastating. As you mentioned, we currently are tracking improvements to $lookup through &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21312&quot; title=&quot;$lookup should batch query requests&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21312&quot;&gt;SERVER-21312&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21284&quot; title=&quot;$lookup should cache query results&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21284&quot;&gt;SERVER-21284&lt;/a&gt;. The best place to track these improvements will be on those tickets and I encourage you to watch and vote on those tickets. I will close this ticket as a duplicate of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21312&quot; title=&quot;$lookup should batch query requests&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21312&quot;&gt;SERVER-21312&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-21284&quot; title=&quot;$lookup should cache query results&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-21284&quot;&gt;SERVER-21284&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;If you need further assistance troubleshooting your performance issues, I encourage you to ask our community for help by posting on the &lt;a href=&quot;https://community.mongodb.com&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;MongoDB Developer Community Forums&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Best,&lt;br/&gt;
Edwin&lt;/p&gt;</comment>
                            <comment id="4382690" author="JIRAUSER1265203" created="Tue, 1 Mar 2022 08:44:50 +0000"  >&lt;p&gt;We had similar performance issues on a cloud project as well. We had to use a different db for that kind of procedures. We would love to go full mongo..watching this..&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="237722">SERVER-21284</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="237997">SERVER-21312</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 1 Mar 2022 08:44:50 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 46 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>edwin.zhou@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 46 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>dimitriskalantzis@gmail.com</customfieldvalue>
            <customfieldvalue>edwin.zhou@mongodb.com</customfieldvalue>
            <customfieldvalue>paplabros@gmail.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0lfxj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i04hx0:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;Create a collection with 100K documents.&lt;/p&gt;

&lt;p&gt;Create a collection with any number of documents.&lt;/p&gt;

&lt;p&gt;Join the second collection on the first collection.&lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                    <customfieldvalue><![CDATA[edwin.zhou@mongodb.com]]></customfieldvalue>
    

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0l22v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>