<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:00:37 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-42483] $multiply operations are slow.</title>
                <link>https://jira.mongodb.org/browse/SERVER-42483</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;When using `$multiply` inside of an aggregate, the execution time of a query is increased dramatically.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;For example, on a 10,000 item match, it is up to 30x slower than the same aggregate with no `$multiply` calculations. For example, I am including two queries with the only difference being using or not using `multiply`. I would expect the calculation to be slower, but not by this order of magnitude.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;I have the queries stored in a file but I am not sure where to attach or send those files.&lt;/p&gt;</description>
                <environment>Mongoose: 5.4.14</environment>
        <key id="878842">SERVER-42483</key>
            <summary>$multiply operations are slow.</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="4">Incomplete</resolution>
                                        <assignee username="daniel.hatcher@mongodb.com">Danny Hatcher</assignee>
                                    <reporter username="rmcquen@funko.com">Ryan McQuen</reporter>
                        <labels>
                    </labels>
                <created>Mon, 29 Jul 2019 20:53:04 +0000</created>
                <updated>Fri, 6 Dec 2019 20:21:04 +0000</updated>
                            <resolved>Fri, 6 Dec 2019 20:21:04 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="2590505" author="daniel.hatcher" created="Fri, 6 Dec 2019 20:20:55 +0000"  >&lt;p&gt;Sorry to hear that you were unable to reproduce but glad to hear that you&apos;re not seeing the issue anymore.&lt;/p&gt;</comment>
                            <comment id="2590487" author="rmcquen@funko.com" created="Fri, 6 Dec 2019 20:09:34 +0000"  >&lt;p&gt;Daniel, unfortunately, no.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;We can close this bug and I will reference it when I get around to making a Mongoose bug. Thank you for the help!&lt;/p&gt;</comment>
                            <comment id="2590461" author="daniel.hatcher" created="Fri, 6 Dec 2019 19:58:53 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=rmcquen%40funko.com&quot; class=&quot;user-hover&quot; rel=&quot;rmcquen@funko.com&quot;&gt;rmcquen@funko.com&lt;/a&gt;, have you had time to check?&lt;/p&gt;</comment>
                            <comment id="2526110" author="rmcquen@funko.com" created="Fri, 8 Nov 2019 20:42:08 +0000"  >&lt;p&gt;Daniel, sorry, I have been very busy and have not had a chance to run this through Mongoose yet. I will try to make time in our next sprint.&lt;/p&gt;</comment>
                            <comment id="2526029" author="daniel.hatcher" created="Fri, 8 Nov 2019 20:24:18 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=rmcquen%40funko.com&quot; class=&quot;user-hover&quot; rel=&quot;rmcquen@funko.com&quot;&gt;rmcquen@funko.com&lt;/a&gt;, have you been able to see this issue again?&lt;/p&gt;</comment>
                            <comment id="2487020" author="daniel.hatcher" created="Thu, 17 Oct 2019 15:54:22 +0000"  >&lt;p&gt;To clarify, if you run those same explain commands via Mongoose do you see the same data or is it significantly worse?&lt;/p&gt;</comment>
                            <comment id="2480905" author="rmcquen@funko.com" created="Mon, 14 Oct 2019 19:56:34 +0000"  >&lt;p&gt;Hmmm, maybe I need to increase the sample size beyond 2000 then. I am also using Mongoose, so maybe there is another factor at play here?&lt;/p&gt;</comment>
                            <comment id="2480892" author="daniel.hatcher" created="Mon, 14 Oct 2019 19:48:00 +0000"  >&lt;p&gt;From the two explain outputs, it appears that the query that didn&apos;t apply any &lt;tt&gt;$multiply&lt;/tt&gt; took 61 milliseconds while the query that performed 10,000 if-then &lt;tt&gt;$multiply&lt;/tt&gt; took 99 milliseconds. Does this match your interpretation? If that is accurate, I don&apos;t believe there&apos;s a bug here. &lt;/p&gt;</comment>
                            <comment id="2476688" author="rmcquen@funko.com" created="Thu, 10 Oct 2019 22:08:06 +0000"  >&lt;p&gt;Actually, I got it under 150 with XZ compression:&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;nobr&quot;&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/attachment/233426/233426_diagnostic.data.tar.xz&quot; title=&quot;diagnostic.data.tar.xz attached to SERVER-42483&quot;&gt;diagnostic.data.tar.xz&lt;sup&gt;&lt;img class=&quot;rendericon&quot; src=&quot;https://jira.mongodb.org/images/icons/link_attachment_7.gif&quot; height=&quot;7&quot; width=&quot;7&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;</comment>
                            <comment id="2476679" author="rmcquen@funko.com" created="Thu, 10 Oct 2019 22:04:10 +0000"  >&lt;p&gt;Daniel, I&apos;ve attached the explains here but the diagnostic data is too large for the 150mb limit, is there some other way I can share it to you?&lt;/p&gt;</comment>
                            <comment id="2426557" author="rmcquen@funko.com" created="Fri, 20 Sep 2019 16:10:36 +0000"  >&lt;p&gt;Daniel, sorry for the late reply, working on getting approval to share that data.&lt;/p&gt;</comment>
                            <comment id="2407709" author="daniel.hatcher" created="Wed, 4 Sep 2019 19:04:32 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=rmcquen%40funko.com&quot; class=&quot;user-hover&quot; rel=&quot;rmcquen@funko.com&quot;&gt;rmcquen@funko.com&lt;/a&gt;, have you had a chance to read my previous comment?&lt;/p&gt;</comment>
                            <comment id="2358290" author="daniel.hatcher" created="Mon, 5 Aug 2019 19:00:30 +0000"  >&lt;p&gt;Could you please provide the &lt;a href=&quot;https://docs.mongodb.com/manual/reference/method/db.collection.aggregate/#return-information-on-aggregation-pipeline-operation&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;explain(true)&lt;/a&gt; outputs for both fast and slow calculations around your inferred breaking point? Additionally, please attach your &quot;diagnostic.data&quot; directory (located under your &lt;tt&gt;$dbpath&lt;/tt&gt;) to this ticket so I can confirm that it&apos;s not a system issue.&lt;/p&gt;</comment>
                            <comment id="2351193" author="rmcquen@funko.com" created="Tue, 30 Jul 2019 20:02:12 +0000"  >&lt;p&gt;Daniel, thanks for the reply! The tipping point seems to be around 2000 items, at which point the calculation (even a calculation of multiplying by 1), takes about twice as long when being processed by multiply. On my system the difference is ~250ms vs ~500ms.&lt;/p&gt;</comment>
                            <comment id="2351023" author="daniel.hatcher" created="Tue, 30 Jul 2019 18:52:10 +0000"  >&lt;p&gt;Adding a large amount of equations to perform on top of a query that didn&apos;t otherwise do those equations will always slow things down. The exact decrease of the slowdown will be dependent on many things including the environment in which the query is run. Do you see a performance decrease of this scale when performing a small amount of multiplies?&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="233426" name="diagnostic.data.tar.xz" size="136748820" author="rmcquen@funko.com" created="Thu, 10 Oct 2019 22:08:18 +0000"/>
                            <attachment id="233423" name="fast_query_explained.txt" size="3887013" author="rmcquen@funko.com" created="Thu, 10 Oct 2019 22:03:08 +0000"/>
                            <attachment id="225547" name="fast_query_no_multiply.json" size="390566" author="rmcquen@funko.com" created="Mon, 29 Jul 2019 20:53:31 +0000"/>
                            <attachment id="233424" name="slow_query_explained.txt" size="12677135" author="rmcquen@funko.com" created="Thu, 10 Oct 2019 22:03:15 +0000"/>
                            <attachment id="225546" name="slow_query_with_multiply.json" size="1380609" author="rmcquen@funko.com" created="Mon, 29 Jul 2019 20:53:31 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>15.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 30 Jul 2019 18:52:10 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 9 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>daniel.hatcher@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 9 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>daniel.hatcher@mongodb.com</customfieldvalue>
            <customfieldvalue>rmcquen@funko.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hvi2av:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hv72bb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hvhok7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>