<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:40:53 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-16385] significant regression in $push performance in 2.6.5 compared to 2.4.3</title>
                <link>https://jira.mongodb.org/browse/SERVER-16385</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I have been doing some benchmarking ahead of the migration of our live system from 2.4.3 to 2.6.5. The first item for concern that my testing has highlighted is that there appears to have been a significant regression in performance in Mongo&apos;s ability to cope with multiple pushes of subdocuments. &lt;br/&gt;
My test scenario is as follows:&lt;br/&gt;
I have two identically configured servers in the same availability zone in AWS (of instance type m1.large). One is running 2.4.3 and the other is 2.6.5. &lt;br/&gt;
I have another larger box (hi1.4xlarge) in the same availability zone from which I execute the test first on one database, then the other.&lt;br/&gt;
The test involves making 100k updates with a push of a single small subdocument to the same collection and measuring the time it takes when pushing 20, 100, 1000, 5000, 10000 subdocs per doc. The collection has several indices including a unique index on a UUID field. &lt;br/&gt;
The test proceeds as follows:&lt;br/&gt;
empty the collection&lt;br/&gt;
generate appropriate number of UUIDs across which to spread 100k subdocs i.e. 100k / 20 = 5000 for the first test&lt;br/&gt;
make 100k updates (with upsert=True) as fast as possible and log time taken / database stats&lt;/p&gt;

&lt;p&gt;From my scatter plot of results, it seems that the test runs fastest with 100 subdocs per doc (presumably because of the cost of updating the indices on the upsert of the larger number of documents dominates the time taken for the push with 20 subdocs per doc). Also it is clear that performance in 2.6.5 has been improved at this end of the scale. &lt;br/&gt;
However, as the number of pushes per document is increased, the performance of 2.6.5 degrades much faster than 2.4.3. I tried both Mongo versions with and without UsePowerOf2Sizes but found the effect of this parameter to be negligible (which was surprising having read the documentation on this feature). Are there any other parameters that affect the allocation of space that could be negatively affecting the performance of 2.6.5 in this test?&lt;/p&gt;

&lt;p&gt;I attach scatter plot of timings, server stats from 2.4.3 and 2.6.5 when pushing 10k subdocs per doc and the python code used for testing (minus all the data logging stuff). &lt;/p&gt;</description>
                <environment></environment>
        <key id="172231">SERVER-16385</key>
            <summary>significant regression in $push performance in 2.6.5 compared to 2.4.3</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-query-execution">Backlog - Query Execution</assignee>
                                    <reporter username="johng">John Greenall</reporter>
                        <labels>
                            <label>query-44-grooming</label>
                    </labels>
                <created>Tue, 2 Dec 2014 14:06:19 +0000</created>
                <updated>Tue, 6 Dec 2022 04:58:31 +0000</updated>
                                            <version>2.6.5</version>
                                                    <component>Performance</component>
                    <component>Write Ops</component>
                                        <votes>1</votes>
                                    <watches>12</watches>
                                                                                                                <comments>
                            <comment id="781787" author="acm" created="Tue, 9 Dec 2014 15:00:31 +0000"  >&lt;p&gt;I&apos;m happy to hear that this isn&apos;t a showstopper for you, and if you find other performance regressions during your testing we of course would be interested in hearing your findings.&lt;/p&gt;

&lt;p&gt;I&apos;m going to put this ticket the &apos;2.9 desired&apos; category, which means we that we intend to address it in the next release after 2.8.&lt;/p&gt;</comment>
                            <comment id="781725" author="johng" created="Tue, 9 Dec 2014 13:56:29 +0000"  >&lt;p&gt;Andrew thanks for the update. This is most helpful. &lt;/p&gt;

&lt;p&gt;I don&apos;t think this issue is on its own a show-stopper for us. We currently do quite a lot of this kind of pushing of subDocuments but we have reworked the pipeline for an upcoming release to do some aggregation of subDocuments before they get inserted into mongoDB as we already had this flagged as performance bottleneck. The rework of the pipeline depends on $min / $max operators and hence we have been hanging fire with pushing this release out until we were happy that 2.6 was stable.&lt;/p&gt;

&lt;p&gt;I&apos;ll continue with benchmarking now I&apos;m happy that you&apos;ve been able to verify my findings and as long as performance elsewhere hasn&apos;t regressed elsewhere we should still be able to proceed with our migration to 2.6.&lt;/p&gt;

&lt;p&gt;Certainly though the preallocation of space for subdocuments is an area where we&apos;d like to either see a bit more built-in intelligence or else be given a bit more control. As stated, the powerOf2Sizes didn&apos;t seem to have much impact for me. &lt;/p&gt;</comment>
                            <comment id="781715" author="acm" created="Tue, 9 Dec 2014 13:39:50 +0000"  >&lt;p&gt;Hi John -&lt;/p&gt;

&lt;p&gt;We are able to reproduce a performance regression here between 2.4 and 2.6. I also tested against 2.8-rc2, which exhibits the same performance regression. We have not yet determined the root cause of the performance regression, but it seems likely that it is related to the refactoring of the update subsystem undertaken in 2.6.&lt;/p&gt;

&lt;p&gt;We will need to undertake further investigation to determine a root cause and understand how to fix it, as well as evaluate the risk of applying such a backport to the 2.6 release, which is now quite stable. It would be helpful if you could provide us with some information about how critical a fix to this issue is for you.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Andrew&lt;/p&gt;


&lt;p&gt;Thanks,&lt;br/&gt;
Andrew&lt;/p&gt;</comment>
                            <comment id="781695" author="johng" created="Tue, 9 Dec 2014 12:30:17 +0000"  >&lt;p&gt;Any preliminary thoughts on this issue guys? I have more testing I&apos;d like to do but don&apos;t want to push on with it until I have some feedback on this in case you&apos;re unable to recreate my numbers and it turns out there is something flawed with my test setup.&lt;br/&gt;
Best,&lt;br/&gt;
John&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="58377" name="243noPow2.svg" size="245747" author="johng" created="Tue, 2 Dec 2014 14:06:19 +0000"/>
                            <attachment id="58378" name="243usePow2.svg" size="235817" author="johng" created="Tue, 2 Dec 2014 14:06:19 +0000"/>
                            <attachment id="58379" name="265usePow2.svg" size="320619" author="johng" created="Tue, 2 Dec 2014 14:06:19 +0000"/>
                            <attachment id="58380" name="mongoSubdocumentPushJira.py" size="2927" author="johng" created="Tue, 2 Dec 2014 14:06:19 +0000"/>
                            <attachment id="58381" name="noPow265.svg" size="308055" author="johng" created="Tue, 2 Dec 2014 14:06:19 +0000"/>
                            <attachment id="58382" name="testTimings.svg" size="48433" author="johng" created="Tue, 2 Dec 2014 14:06:19 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25125"><![CDATA[Query Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 9 Dec 2014 13:39:50 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        9 years, 10 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            9 years, 10 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>andrew.morrow@mongodb.com</customfieldvalue>
            <customfieldvalue>backlog-query-execution</customfieldvalue>
            <customfieldvalue>johng</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlhkf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr2h0n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>150773</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsa01b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>