<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:18:02 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-28405] Performance degradation 3.4.2 vs 3.2.11</title>
                <link>https://jira.mongodb.org/browse/SERVER-28405</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Hello,&lt;/p&gt;

&lt;p&gt;We have a rather large mongo replica set containing 19,000 dbs. We have an outstanding issue with wiredTiger cache as per &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-27700&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-27700&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-27753&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-27753&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;As we are expecting the change to be in the next 3.4.x-rc we upgraded our cluster from 3.2.11 to 3.4.2 to understand the current performance characteristics and hope that we would gain some improvement.&lt;/p&gt;

&lt;p&gt;Since the upgrade (less than 24 hours) our jobs (which use this cluster) are now taking many hours to finish. In fact some of them have not finished running and we have canceled them.&lt;/p&gt;

&lt;p&gt;The following a screen shot of job run times and how mongo 3.2.11 / 3.4.2 impacts the run time. &lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;a id=&quot;152018_thumb&quot; href=&quot;https://jira.mongodb.org/secure/attachment/152018/152018_Screen+Shot+2017-03-21+at+09.49.22.png&quot; title=&quot;Screen Shot 2017-03-21 at 09.49.22.png&quot; file-preview-type=&quot;image&quot; file-preview-id=&quot;152018&quot; file-preview-title=&quot;Screen Shot 2017-03-21 at 09.49.22.png&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/thumbnail/152018/_thumb_152018.png&quot; style=&quot;border: 0px solid black&quot; role=&quot;presentation&quot;/&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;


&lt;p&gt;Please can you provide an upload link and I will provide the diagnostic data for the cluster.&lt;/p&gt;

&lt;p&gt;Kind regards,&lt;br/&gt;
Luke&lt;/p&gt;
</description>
                <environment></environment>
        <key id="366509">SERVER-28405</key>
            <summary>Performance degradation 3.4.2 vs 3.2.11</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13203">Gone away</resolution>
                                        <assignee username="bruce.lucas@mongodb.com">Bruce Lucas</assignee>
                                    <reporter username="lukemorfitt">Luke Morfitt</reporter>
                        <labels>
                    </labels>
                <created>Tue, 21 Mar 2017 09:56:47 +0000</created>
                <updated>Fri, 27 Oct 2023 20:44:10 +0000</updated>
                            <resolved>Mon, 3 Apr 2017 19:52:56 +0000</resolved>
                                    <version>3.4.2</version>
                                                    <component>WiredTiger</component>
                                        <votes>0</votes>
                                    <watches>9</watches>
                                                                                                                <comments>
                            <comment id="1536759" author="lukemorfitt" created="Thu, 30 Mar 2017 09:01:14 +0000"  >&lt;p&gt;Hi,&lt;/p&gt;

&lt;p&gt;Sorry for the delay getting back to you. &lt;/p&gt;

&lt;p&gt;As 3.4.3 has been released I would like to close this ticket and retest. 3.4.3 has the performance fix we have been waiting for.&lt;/p&gt;

&lt;p&gt;Thank you for your continued support,&lt;br/&gt;
Luke&lt;/p&gt;</comment>
                            <comment id="1530697" author="bruce.lucas@10gen.com" created="Wed, 22 Mar 2017 20:12:39 +0000"  >&lt;p&gt;Hi Luke,&lt;/p&gt;

&lt;p&gt;I&apos;d like to add another question: your chart shows a restart under 3.2.11 at 20.03 00:00, but I&apos;m not seeing that in the logs or diagnostic data you uploade. Can you clarify what node was restarted?&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Bruce&lt;/p&gt;</comment>
                            <comment id="1530671" author="bruce.lucas@10gen.com" created="Wed, 22 Mar 2017 19:53:17 +0000"  >&lt;p&gt;Hi Luke,&lt;/p&gt;

&lt;p&gt;Thanks for uploading the data, we&apos;re taking a look.&lt;/p&gt;

&lt;p&gt;I&apos;d like to correlate the events in your chart with our data. Can you clarify:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;What is the timezone of the times shown on your chart?&lt;/li&gt;
	&lt;li&gt;I think each of the triangles on your chart is a single job and the line represents cumulative time for that job so the height represents the total time for the job, is that correct?&lt;/li&gt;
	&lt;li&gt;The jobs that started prior to about 20:45 under 3.4.2 finished in 30-60 minutes, which is similar to the time you are seeing under 3.2.11 with a full cache, whereas you are looking for times under 30 minutes, like you see after a restart under 3.2.11, am I understanding correctly?&lt;/li&gt;
	&lt;li&gt;The job that started at about 20:45 took many hours, and that cannot be explained by the job doing more work, correct?&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Regarding the jobs can you clarify&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;How much variability is inherent in the size of the jobs? I understand that variability in the jobs does not account for the slowness you are seeing, but I am interested to maybe correlate job run time with the internal state in mongod, but to do that I need to know how much variability is inherent in the jobs themselves.&lt;/li&gt;
	&lt;li&gt;Do the jobs do any work on the secondaries, or do they depend only on the primary? This dependency could include both direct dependency, like secondary queries, or indirect dependency, like writes with majority write concern. We noticed that there are some secondary queries and also some secondary lag, and want to understand what impact secondary performance might have on the job runtime issue you are seeing.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Thanks,&lt;br/&gt;
Bruce&lt;/p&gt;</comment>
                            <comment id="1529257" author="lukemorfitt" created="Tue, 21 Mar 2017 14:37:21 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=anonymous.user&quot; class=&quot;user-hover&quot; rel=&quot;anonymous.user&quot;&gt;anonymous.user&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thank you. I&apos;ve uploaded the data from all the nodes in the replica set.&lt;/p&gt;

&lt;p&gt;Kind regards,&lt;br/&gt;
Luke&lt;/p&gt;</comment>
                            <comment id="1529229" author="thomas.schubert" created="Tue, 21 Mar 2017 14:13:37 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lukemorfitt&quot; class=&quot;user-hover&quot; rel=&quot;lukemorfitt&quot;&gt;lukemorfitt&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;I&apos;ve created a &lt;a href=&quot;https://10gen-httpsupload.s3.amazonaws.com/upload_forms/5cf12e34-082e-45da-b7f6-304a6f3f8502.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;secure portal&lt;/a&gt;, would you please upload the the complete log files as well as the &lt;tt&gt;diagnostic.data&lt;/tt&gt;?&lt;/p&gt;

&lt;p&gt;Thank you,&lt;br/&gt;
Thomas&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="152018" name="Screen Shot 2017-03-21 at 09.49.22.png" size="75946" author="lukemorfitt" created="Tue, 21 Mar 2017 09:52:06 +0000"/>
                            <attachment id="152165" name="event.png" size="435920" author="bruce.lucas@mongodb.com" created="Wed, 22 Mar 2017 20:52:37 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 21 Mar 2017 14:13:37 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 45 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 45 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>bruce.lucas@mongodb.com</customfieldvalue>
            <customfieldvalue>kelsey.schubert@mongodb.com</customfieldvalue>
            <customfieldvalue>lukemorfitt</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht4jef:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hswzgf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                    <customfieldvalue><![CDATA[bruce.lucas@mongodb.com]]></customfieldvalue>
    

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hs4dsv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>