<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:15:37 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-27596] Wiredtiger very high CPU usage</title>
                <link>https://jira.mongodb.org/browse/SERVER-27596</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I&apos;m running a 3 member replicaset. The average workload is very constant and does not push load average above 2-3. When workload increases (for example due to a delay in our processing queue) mongodb starts processing queries very fast but after about 5-10 minutes completely starts hogging userspace CPU usage (100%, load average 120-130).&lt;/p&gt;

&lt;p&gt;Even when completely shutting down the workload, load average still stays at 120-130, the only way for it to go back to normal is running rs.stepDown() and force another member to be primary. This instance then (usually) starts processing the requests very fast but sometimes still goes back to the very high load average.&lt;/p&gt;

&lt;p&gt;In the mongodb log I can see write requests are taking a very long time (some upto 10 seconds).&lt;/p&gt;

&lt;p&gt;I&apos;m not really sure how to prevent this as it completely disables my whole workload.&lt;/p&gt;</description>
                <environment>Debian 7, xeon d1521, 32gb ram, 2x480gb ssd, 1gbit networking</environment>
        <key id="343193">SERVER-27596</key>
            <summary>Wiredtiger very high CPU usage</summary>
                <type id="6" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14720&amp;avatarType=issuetype">Question</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="kelsey.schubert@mongodb.com">Kelsey Schubert</assignee>
                                    <reporter username="vincentvm">Vincent van Megen</reporter>
                        <labels>
                    </labels>
                <created>Fri, 6 Jan 2017 11:03:20 +0000</created>
                <updated>Mon, 30 Jan 2017 17:37:41 +0000</updated>
                            <resolved>Mon, 30 Jan 2017 17:37:41 +0000</resolved>
                                    <version>3.2.11</version>
                                                    <component>WiredTiger</component>
                                        <votes>0</votes>
                                    <watches>8</watches>
                                                                                                                <comments>
                            <comment id="1488253" author="thomas.schubert" created="Mon, 30 Jan 2017 17:37:29 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=vincentvm&quot; class=&quot;user-hover&quot; rel=&quot;vincentvm&quot;&gt;vincentvm&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;It appears that your workload is hitting hardware limits. The workload increase in is significant (possibly as high as 10x as many operations at its peak) and it&apos;s likely that the system&apos;s I/O was initially constrained as the workload spiked. As throughput slows, more work continues to come into applications, but cannot be cleared as quickly, causing WiredTiger cache eviction threads to hit the CPU limit. My advice would be to consider stabilizing your workload or provision a more powerful host to resolve this issue.&lt;/p&gt;

&lt;p&gt;Kind regards,&lt;br/&gt;
Thomas&lt;/p&gt;</comment>
                            <comment id="1472433" author="vincentvm" created="Mon, 9 Jan 2017 12:45:39 +0000"  >&lt;p&gt;Another screenshot of MongoDB Compass showing the amount of queued writes increasing.&lt;/p&gt;</comment>
                            <comment id="1472425" author="vincentvm" created="Mon, 9 Jan 2017 12:16:57 +0000"  >&lt;p&gt;2 screenshots of cloud.mongodb.com statistics while this issue is happening.&lt;/p&gt;</comment>
                            <comment id="1471283" author="vincentvm" created="Fri, 6 Jan 2017 16:57:44 +0000"  >&lt;p&gt;I don&apos;t have the logs anymore, i uploaded the diagnostics data directory. It&apos;s in 3 folders 0,1,2 directory 2 was the primary servers. It started happening at around 9am gmt+1 and lasted untill about 11am gmt+1 on the 6th of january.&lt;/p&gt;</comment>
                            <comment id="1471272" author="thomas.schubert" created="Fri, 6 Jan 2017 16:41:40 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=vincentvm&quot; class=&quot;user-hover&quot; rel=&quot;vincentvm&quot;&gt;vincentvm&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;If you could upload the complete directory, it may provide us with additional context about your typical workload. I&apos;ve created a &lt;a href=&quot;https://10gen-httpsupload.s3.amazonaws.com/upload_forms/425f90df-196f-416b-afaf-0aba44be157c.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;secure portal&lt;/a&gt; for you to upload the &lt;tt&gt;diagnostic.data&lt;/tt&gt; as well as the complete mongod logs for each node.&lt;/p&gt;

&lt;p&gt;Thanks again,&lt;br/&gt;
Thomas&lt;/p&gt;</comment>
                            <comment id="1471265" author="vincentvm" created="Fri, 6 Jan 2017 16:36:55 +0000"  >&lt;p&gt;I have the diagnostics.data directory, where can I upload these files? Should I just upload the diagnostics files from the day this happened?&lt;/p&gt;</comment>
                            <comment id="1471242" author="thomas.schubert" created="Fri, 6 Jan 2017 16:18:11 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=vincentvm&quot; class=&quot;user-hover&quot; rel=&quot;vincentvm&quot;&gt;vincentvm&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thanks for reporting this behavior. So we can continue to investigate, would you please provide an archive of the &lt;tt&gt;diagnostic.data&lt;/tt&gt; and complete logs for each node in the replica set?&lt;/p&gt;

&lt;p&gt;Thank you,&lt;br/&gt;
Thomas&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="147178" name="Screen Shot 2017-01-06 at 11.59.28.png" size="425929" author="vincentvm" created="Fri, 6 Jan 2017 11:00:26 +0000"/>
                            <attachment id="147249" name="Screen Shot 2017-01-09 at 13.15.39.png" size="169293" author="vincentvm" created="Mon, 9 Jan 2017 12:16:34 +0000"/>
                            <attachment id="147250" name="Screen Shot 2017-01-09 at 13.15.49.png" size="198591" author="vincentvm" created="Mon, 9 Jan 2017 12:16:33 +0000"/>
                            <attachment id="147251" name="Screen Shot 2017-01-09 at 13.44.14.png" size="72611" author="vincentvm" created="Mon, 9 Jan 2017 12:44:59 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 6 Jan 2017 16:18:11 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        7 years, 2 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>kelsey.schubert@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            7 years, 2 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>kelsey.schubert@mongodb.com</customfieldvalue>
            <customfieldvalue>vincentvm</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht0k33:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hstgsv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                    <customfieldvalue><![CDATA[kelsey.schubert@mongodb.com]]></customfieldvalue>
    

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlkif:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>