<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:51:08 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-19485] Cpu 100% when mongodb running with normal ops</title>
                <link>https://jira.mongodb.org/browse/SERVER-19485</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Hi Team , &lt;/p&gt;

&lt;p&gt;There is a mongodb session will cost cpu 100%  when there is only normal ops on the instance , issue can be solved by restart the database , but it will appear after a few weeks .&lt;br/&gt;
It is always the same pid in linux costs 100% cpu , attached some screen shot while running top command .&lt;/p&gt;

&lt;p&gt;As well I see the information : &quot;thread over memory limit, cleaning up&quot; very often in the db log , there is an issue with my database ?&lt;/p&gt;

&lt;p&gt;Config file option: &lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;processManagement :&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; fork : true&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; pidFilePath :  /home/Radius/Rad.pid&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;systemLog:&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; destination : file&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; path : /home/Radius/Log/Rad_Log&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; logAppend : true&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;storage:&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; dbPath : /home/Radius/Data_WiredTiger&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; engine : wiredTiger&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; wiredTiger :&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  engineConfig :&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;   cacheSizeGB : 20&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; directoryPerDB : true&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;net:&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; port : 40001&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; maxIncomingConnections : 2000&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;security:&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; authorization  : enabled&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; keyFile : /home/Radius/Keys/RadKey&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;replication:&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; replSetName : rad&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;auditLog:&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; destination : file&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; format : JSON&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; path : /home/Radius/Log/RadiusDBAduit.json&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; filter : &apos;{ atype : {$nin : [&quot;authenticate&quot;]} }&apos;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;</description>
                <environment></environment>
        <key id="217513">SERVER-19485</key>
            <summary>Cpu 100% when mongodb running with normal ops</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="ramon.fernandez@mongodb.com">Ramon Fernandez Marina</assignee>
                                    <reporter username="carl.dong@windfindtech.com">carl dong</reporter>
                        <labels>
                    </labels>
                <created>Mon, 20 Jul 2015 03:35:47 +0000</created>
                <updated>Sat, 26 Sep 2015 12:20:19 +0000</updated>
                            <resolved>Sat, 26 Sep 2015 12:20:19 +0000</resolved>
                                    <version>3.0.4</version>
                                                    <component>Performance</component>
                    <component>WiredTiger</component>
                                        <votes>4</votes>
                                    <watches>13</watches>
                                                                                                                <comments>
                            <comment id="1044200" author="ramon.fernandez" created="Sat, 26 Sep 2015 12:20:19 +0000"  >&lt;p&gt;Closing this ticket after the 3.0.6 release has been out for a while and no new reports of this behavior have come in.&lt;/p&gt;</comment>
                            <comment id="1018052" author="ramon.fernandez" created="Mon, 31 Aug 2015 11:07:12 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=huy&quot; class=&quot;user-hover&quot; rel=&quot;huy&quot;&gt;huy&lt;/a&gt;, can you please open a separate ticket? Even if you&apos;re seeing the same symptoms the root cause may be different. Please provide full details about your deployment (replica set configuration, amount of memory, number of cpus, etc.) and full logs for the affected server. The first order of business will probably be to collect server statistics for a period of time &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-19485?focusedCommentId=988736&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-988736&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;as described above&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                            <comment id="1018042" author="huy" created="Mon, 31 Aug 2015 10:38:30 +0000"  >&lt;p&gt;Hi everybody,&lt;/p&gt;

&lt;p&gt;I follow this issue because I&apos;m facing with the same problem. (mongo 3.0.6)&lt;br/&gt;
My server status when it happened (file attach).&lt;/p&gt;

&lt;p&gt;Regards&lt;/p&gt;

&lt;p&gt;Huy.&lt;/p&gt;</comment>
                            <comment id="1016198" author="ramon.fernandez" created="Thu, 27 Aug 2015 17:58:37 +0000"  >&lt;p&gt;Thanks &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=carl.dong%40windfindtech.com&quot; class=&quot;user-hover&quot; rel=&quot;carl.dong@windfindtech.com&quot;&gt;carl.dong@windfindtech.com&lt;/a&gt;. We&apos;ll keep this ticket in &quot;Waiting for User Input&quot; until we hear back from you.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                            <comment id="1015756" author="carl.dong@windfindtech.com" created="Thu, 27 Aug 2015 07:49:43 +0000"  >&lt;p&gt;Ramon ,  I have updated my database to MongoDB 3.0.6 , will monitor it for some days and keep here update.&lt;/p&gt;</comment>
                            <comment id="1014476" author="ramon.fernandez" created="Tue, 25 Aug 2015 23:04:12 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=carl.dong%40windfindtech.com&quot; class=&quot;user-hover&quot; rel=&quot;carl.dong@windfindtech.com&quot;&gt;carl.dong@windfindtech.com&lt;/a&gt;, as per &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=michael.cahill&quot; class=&quot;user-hover&quot; rel=&quot;michael.cahill&quot;&gt;michael.cahill&lt;/a&gt;&apos;s comment above &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-19522&quot; title=&quot;Capped collection insert rate declines over time under WiredTiger&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-19522&quot;&gt;&lt;del&gt;SERVER-19522&lt;/del&gt;&lt;/a&gt; was fixed in MongoDB 3.0.6, which was released earlier this week. Can you please try 3.0.6 and see if it addresses your issue?&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                            <comment id="991643" author="michael.cahill" created="Wed, 29 Jul 2015 07:17:09 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=carl.dong%40windfindtech.com&quot; class=&quot;user-hover&quot; rel=&quot;carl.dong@windfindtech.com&quot;&gt;carl.dong@windfindtech.com&lt;/a&gt; we have been looking through the data, the best theory we have at the moment is that you might be hitting &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-19522&quot; title=&quot;Capped collection insert rate declines over time under WiredTiger&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-19522&quot;&gt;&lt;del&gt;SERVER-19522&lt;/del&gt;&lt;/a&gt;, which has a fix in progress.&lt;/p&gt;

&lt;p&gt;The symptom of that ticket is that truncating the oplog to keep the size capped can leave behind some empty pages in the cache.  Over time, the number of pages can grow large and when a checkpoint runs after the oplog is truncated, it has to step over all of those empty pages before getting useful work done.&lt;/p&gt;

&lt;p&gt;I will let you know when the issue is resolved: we are currently testing the fix.&lt;/p&gt;</comment>
                            <comment id="990323" author="carl.dong@windfindtech.com" created="Tue, 28 Jul 2015 02:50:55 +0000"  >&lt;p&gt;Bruce ,&lt;/p&gt;

&lt;p&gt;1. There is no regular periodic application operating on the database. &lt;br/&gt;
2. I believe it happens much more often under heavy load , and it should be something accumulation trigger the issue  , cause  as I restarted the database , under the same level load it is not happening in many days .&lt;br/&gt;
3. I can only provide the log with level 2  while the issue happening in the database , there is not information in /var/log/messages or  /var/log/syslog when when issue happen.&lt;br/&gt;
4. There are 40 cpus on the server , and the 100% cpu means only 1 cpu is 100% used ( running __wt_tree_walk ), others are in normal state.&lt;/p&gt;

&lt;p&gt;Thanks&lt;/p&gt;

&lt;p&gt;Carl &lt;/p&gt;</comment>
                            <comment id="989499" author="bruce.lucas@10gen.com" created="Mon, 27 Jul 2015 13:38:24 +0000"  >&lt;p&gt;Thanks Carl. From that data we can see that the high CPU utilization precisely correlates with the checkpoints. This suggests that the checkpoint activity is either directly or indirectly responsible for the high CPU utilization. (I&apos;ve attached a graph showing the correlation and some other correlated statistics that we&apos;re using for an internal discussion.)&lt;/p&gt;

&lt;p&gt;The regularity of the occurrence of the issue is striking. I have two theories about that:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Some regular periodic application activity triggers the issue; do you have any such periodic activity?&lt;/li&gt;
	&lt;li&gt;However the interval between occurrences, while regular, seems to be somewhat variable, and possibly load dependent - for example, you mention that during busy times it occurs every 10 minutes, whereas in the data you uploaded it it happened at 20 minute intervals. Do you believe it happens more often under heavy load? This suggests an accumulation of some kind of state that is then periodically cleaned up in some way.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Would you be able to upload the exact mongod log corresponding to the latest data you uploaded? This will allow us to look for events that are exactly correlated with the CPU issue. Also if you could upload the syslog file (/var/log/messages or /var/log/syslog) corresponding to the same time period we could look for correlated events at the system level.&lt;/p&gt;

&lt;p&gt;Finally the CPU percentage reported by iostat was at most about 5.8%, so I assume that the 100% CPU utilization that you mention was 100% of a single CPU on a multi-CPU system (whereas iostat is reporting % of total CPU resources). Since WT in general is able to make effective use of multiple CPUs, while 100% utilization of a single CPU is something to be investigated, I wouldn&apos;t expect it to necessarily cause any issues in a multi-CPU system. Are you observing any application-level issues as a result of the 100% CPU utilization? If so that might tell us more about the nature of the 100% CPU issue.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Bruce&lt;/p&gt;</comment>
                            <comment id="989433" author="mydigitalself" created="Mon, 27 Jul 2015 12:23:04 +0000"  >&lt;p&gt;Michael, the main problem that we&apos;ve had is that we&apos;ve only experienced this issue when the WiredTiger replica members are primary. We&apos;ve now stepped them down to secondary, so not sure if we&apos;re going to be able to get any more data as we&apos;re loathe to step the machines back up again as the problem brings our entire app down to it&apos;s knees when it occurs.&lt;/p&gt;

&lt;p&gt;We will see what we can do. Do you have any specific references to issues fixed in 3.0.5 and is it still on track for release tomorrow?&lt;/p&gt;

&lt;p&gt;Cheers&lt;br/&gt;
Mike&lt;br/&gt;
(PS. How do you mention people in Jira?)&lt;/p&gt;</comment>
                            <comment id="989256" author="carl.dong@windfindtech.com" created="Mon, 27 Jul 2015 03:12:27 +0000"  >&lt;p&gt;Usually the issue will last for 2 - 3 mins ,  in busy time , it happens every 10 mins .&lt;/p&gt;

&lt;p&gt;Just capture 3 times issue happen with serverstatus and iostat, time period:&lt;br/&gt;
02:18:24~02:21:26  ,  02:38:20~02:40:40  ,   02:57:48~02:59:48&lt;br/&gt;
Attache logfiles.&lt;/p&gt;

&lt;p&gt;Bty , I&apos;m going to  upgrade to 3.0.5 as soon as stable release available with the fixed capped collection performance to see if the problem related to the capped collections.&lt;/p&gt;


</comment>
                            <comment id="988736" author="ramon.fernandez" created="Fri, 24 Jul 2015 21:55:58 +0000"  >&lt;p&gt;Thanks for posting that information &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=carl.dong%40windfindtech.com&quot; class=&quot;user-hover&quot; rel=&quot;carl.dong@windfindtech.com&quot;&gt;carl.dong@windfindtech.com&lt;/a&gt;. Based on the times you mention the CPU utilization seems to be correlated with some unusually long checkpoints, but we would need more precise timing information to tell whether it is victim or culprit. Would it be possible for you to repeat the serverStatus timeseries data collection, but also add iostat, as follows:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;mongo --eval &quot;while(true) {print(JSON.stringify(db.serverStatus())); sleep(1000)}&quot; &amp;gt;ss.log&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;iostat -k -t -x 1 &amp;gt;iostat.log &amp;amp;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;Note I have adjust the delay in the serverStatus timeseries to 1s, matching the 1s resolution of the iostat data. If you could do this a couple of times, or leave it running long enough to capture a couple of incidents, that will help us pin down the correlation between CPU utilization and events inside mongod.&lt;/p&gt;

&lt;p&gt;Also, can you tell us what the general pattern of 100% CPU utilization is - how long does the issue last? How frequently does it occur (once it has started occurring after the few weeks)?&lt;/p&gt;

&lt;p&gt;By the way, I should note that in the serverStatus excerpt you posted earlier the oplog is full, as the oplog size limit is based on &quot;size&quot;, not &quot;storageSize&quot;, and size==maxSize in that excerpt. Given that, do you think it is possible that the problem begins to occur after the oplog becomes full?&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Ram&#243;n (on Bruce&apos;s behalf)&lt;/p&gt;</comment>
                            <comment id="987804" author="carl.dong@windfindtech.com" created="Fri, 24 Jul 2015 02:59:40 +0000"  >&lt;p&gt;ss.log for issue happen around  02:24:34.&lt;br/&gt;
ss1.log for issue happed around 02:41:50.&lt;/p&gt;
</comment>
                            <comment id="987746" author="michael.cahill" created="Fri, 24 Jul 2015 01:18:50 +0000"  >&lt;p&gt;Thanks for the additional information &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=mydigitalself&quot; class=&quot;user-hover&quot; rel=&quot;mydigitalself&quot;&gt;mydigitalself&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lalo&quot; class=&quot;user-hover&quot; rel=&quot;lalo&quot;&gt;lalo&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=carl.dong%40windfindtech.com&quot; class=&quot;user-hover&quot; rel=&quot;carl.dong@windfindtech.com&quot;&gt;carl.dong@windfindtech.com&lt;/a&gt;.  At the moment we can&apos;t be completely sure that these three cases all have the same underlying cause but we are making that assumption.  Getting &quot;perf top&quot; output when the system gets into this state would help to confirm this.&lt;/p&gt;

&lt;p&gt;There are some differences: we note that in &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lalo&quot; class=&quot;user-hover&quot; rel=&quot;lalo&quot;&gt;lalo&lt;/a&gt;&apos;s case, the system is using all available memory, whereas in the other cases there appears to be free memory available.  When there is memory pressure, some data to be evicted from RAM managed by Linux (even if swap is disabled).  That may partially explain additional reads.  There are a number of fixes coming in 3.0.5 for issues where MongoDB with WiredTiger could use more memory that expected.&lt;/p&gt;

&lt;p&gt;Are any of you able to gather server stats over a period of time (e.g., 5 minutes) when the system is in this state, then again (for another 5 minutes) after it is restarted and warmed up?  That should help us pinpoint the differences: at the moment we are unable to reproduce this behavior and while we have some theories, we can&apos;t find a clear explanation.&lt;/p&gt;

&lt;p&gt;Here is the recipe for gathering statistics every 10 seconds:&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;mongo --eval &quot;while(true) {print(JSON.stringify(db.serverStatus())); sleep(10000)}&quot; &amp;gt;ss.log&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;</comment>
                            <comment id="986765" author="mydigitalself" created="Thu, 23 Jul 2015 09:31:02 +0000"  >&lt;p&gt;Hi folks,&lt;/p&gt;

&lt;p&gt;We (Gitter) have experienced exactly the same issue twice now and thought I&apos;d chime in a little bit with as much information as I can provide.&lt;/p&gt;

&lt;p&gt;Replicasetup&lt;br/&gt;
1xarbiter&lt;br/&gt;
2xmongo 3.0.4 (WiredTiger/Snappy) (mongo-replica-member-001 &amp;amp; mongo-replica-member-005)&lt;br/&gt;
1xmongo 2.6 (mongo-replica-member-002)&lt;/p&gt;

&lt;p&gt;We kept the 2.6 box specifcally for just-in-case scenarios as we only recently upgraded to WiredTiger.&lt;/p&gt;

&lt;p&gt;The mongo 3.0.4 boxes are are both r3.large AWS instances sitting on top of PIOPS (2000) SSD drives. The 2.6 box is also an r3.large, but is on a RAID-10 array. So I know there are some differences in the infrastructure, but we are trying to move everything to single PIOPS SSD drives as they are much easier to manage.&lt;/p&gt;

&lt;p&gt;Yesterday, mongo-replica-member-005 (WiredTiger) was Primary - we prefer the WiredTiger boxes to be primary, as their performance is much better - well done! - although we are starting to reconsider this given this issue has happened twice in production now and brings everything to it&apos;s knees.&lt;/p&gt;

&lt;p&gt;Unfortunately I don&apos;t have a db.serverStatus() saved from when it happened. What we did notice is that the session open cursor count the first time we saw this issue was ~60k, which seems extraordinarily high to us. Below you can see a paste of doing db.serverStatus. Notice the increase across a minute.&lt;/p&gt;

&lt;p&gt;TroupeReplicaSet:PRIMARY&amp;gt; db.serverStatus().wiredTiger.session&lt;/p&gt;
{ &quot;open cursor count&quot; : 32797, &quot;open session count&quot; : 144 }
&lt;p&gt;TroupeReplicaSet:PRIMARY&amp;gt; db.serverStatus().wiredTiger.session&lt;/p&gt;
{ &quot;open cursor count&quot; : 32809, &quot;open session count&quot; : 144 }

&lt;p&gt;and again this morning (~12 hours later)&lt;/p&gt;

&lt;p&gt;TroupeReplicaSet:SECONDARY&amp;gt; db.serverStatus().wiredTiger.session&lt;/p&gt;
{ &quot;open cursor count&quot; : 46177, &quot;open session count&quot; : 163 }

&lt;p&gt;This may be a red herring.&lt;/p&gt;

&lt;p&gt;The only information I can provide is a screenshot from DataDog.&lt;br/&gt;
&lt;a href=&quot;https://infinit.io/_/W486hu4&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://infinit.io/_/W486hu4&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;There are a few things to notice here:&lt;/p&gt;

&lt;p&gt;1. Sudden increased CPU usage&lt;/p&gt;

&lt;p&gt;An observation I noticed on the box at the top was watching with &quot;top&quot; I could see the mongodb process was running at ~107%. Normally it runs around 7%. Given the box has two cores and our understanding of the threads, that perhaps one thread was operating at 100% and the other was ticking along as normal at 7%. You can sort of see this in the CPU usage graph in that it shoots up to mildly above 50% (i.e. of 2 cores).&lt;/p&gt;

&lt;p&gt;You can also notice there is a gradual increase for a few hours prior to the large spike.&lt;/p&gt;

&lt;p&gt;2. Memory breakdown&lt;/p&gt;

&lt;p&gt;Corresponding to the gradual increase in the CPU before the spike, you can see the memory usage races - effectively doubles. We have a vague recollection of reading something about WiredTiger not consuming all memory at first, but 50% and then increasing this on demand. We could be incorrect, but perhaps this is what&apos;s happening here is that more memory is allocated and then something funny (yes, that is a technical term) starts to happen after a while. &lt;/p&gt;

&lt;p&gt;3. OPS&lt;/p&gt;

&lt;p&gt;The lower right graph shows consumed OPS across all volumes. It&apos;s hard to tell as the purple line covers it a little, but corresponding to the initial increase in CPU and memory jump, the SSD (blue line) takes a little jump for a sustained period and the a huge jump during the CPU spike. You can see on AWS, that this is because of read operations: &lt;a href=&quot;https://infinit.io/_/dn3Qn7t&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://infinit.io/_/dn3Qn7t&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;==&lt;/p&gt;

&lt;p&gt;The drops in the CPU spike and OPS were all caused by us stepping the box down as primary, and after a few minutes everything returned to normal without restarting mongod.&lt;/p&gt;

&lt;p&gt;One final observation...&lt;/p&gt;

&lt;p&gt;lsof reports that mongo has an incredibly high number of open files (at all times)...&lt;/p&gt;

&lt;p&gt;root@mongo-replica-member-005:/home/deployer# lsof | grep data2 | wc -l&lt;br/&gt;
354316&lt;/p&gt;

&lt;p&gt;And our ulimit in mongodb.conf is only 100000....&lt;/p&gt;

&lt;p&gt;==&lt;/p&gt;

&lt;p&gt;I hope that helps. We&apos;re loathe to put the box back as a primary until we have more insight on this, so I&apos;m not sure if it will happen again.&lt;/p&gt;

&lt;p&gt;Question for Lars Lorenzen: you mentioned you saw this on a slave (secondary I assume?), do you perform read operations against this slave at all (slaveOK)? We don&apos;t and so I&apos;m not sure it will exhibit the same behaviour. &lt;/p&gt;

&lt;p&gt;If it does happen again, we will certainly save the db.serverStatus and the results of perf top.&lt;/p&gt;

&lt;p&gt;Any insight that anyone could provide would be great. We would be happy to field calls and work interactively with anyone at Mongo to help pin down this issue as we are all on edge right now &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/wink.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/p&gt;

&lt;p&gt;Cheers&lt;br/&gt;
Mike&lt;/p&gt;
</comment>
                            <comment id="970040" author="lalo" created="Mon, 20 Jul 2015 16:26:48 +0000"  >&lt;p&gt;Can provide the log which is flowing over of slow query warnings. Happened on 2015-07-16 20:00 CEST. &lt;br/&gt;
I did not make screenshots of the CPU usage.&lt;br/&gt;
Will collect information next time it happens.&lt;br/&gt;
We are running mongodb 3.0.4 on CentOS and ext4 ssd.&lt;/p&gt;</comment>
                            <comment id="969947" author="dan@10gen.com" created="Mon, 20 Jul 2015 15:10:20 +0000"  >&lt;p&gt;hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lalo&quot; class=&quot;user-hover&quot; rel=&quot;lalo&quot;&gt;lalo&lt;/a&gt;, are you able to run the same diagnostic steps as Carl has done previously?&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Dan&lt;/p&gt;</comment>
                            <comment id="969943" author="lalo" created="Mon, 20 Jul 2015 15:05:23 +0000"  >&lt;p&gt;Having the same problem on our slave running mongod 3.0.4: One process is using 100% cpu and slowing down the whole database (read performance very poor). Restart helps.&lt;/p&gt;</comment>
                            <comment id="969711" author="carl.dong@windfindtech.com" created="Mon, 20 Jul 2015 07:40:35 +0000"  >&lt;p&gt;Attached serverstatus ,&lt;br/&gt;
There are only normal ops in the instance , after restart the mongod , the issue will gone , I did this before but issue comes again in a few weeks . &lt;/p&gt;</comment>
                            <comment id="969697" author="michael.cahill" created="Mon, 20 Jul 2015 07:19:00 +0000"  >&lt;p&gt;Thanks &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=carl.dong%40windfindtech.com&quot; class=&quot;user-hover&quot; rel=&quot;carl.dong@windfindtech.com&quot;&gt;carl.dong@windfindtech.com&lt;/a&gt;, that makes things a little clearer.  That thread is the one that finds pages to evict from the WiredTiger cache.  It is expected to run frequently, but unless there are a lot of reads, I would not expect it to use 100% of a CPU core.&lt;/p&gt;

&lt;p&gt;Can you please also run &lt;tt&gt;db.serverStatus()&lt;/tt&gt; and post the full output here?&lt;/p&gt;</comment>
                            <comment id="969691" author="carl.dong@windfindtech.com" created="Mon, 20 Jul 2015 07:08:14 +0000"  >&lt;p&gt;As well the oplog.rs collection is not full in my database.&lt;/p&gt;


&lt;p&gt;&amp;gt; db.oplog.rs.stats(1024*1024)&lt;br/&gt;
{&lt;br/&gt;
        &quot;ns&quot; : &quot;local.oplog.rs&quot;,&lt;br/&gt;
        &quot;count&quot; : 188920828,&lt;br/&gt;
        &quot;size&quot; : 51200,&lt;br/&gt;
        &quot;avgObjSize&quot; : 284,&lt;br/&gt;
        &quot;storageSize&quot; : 21962,&lt;br/&gt;
        &quot;capped&quot; : true,&lt;br/&gt;
        &quot;max&quot; : -1,&lt;br/&gt;
        &quot;maxSize&quot; : 51200,&lt;/p&gt;</comment>
                            <comment id="969689" author="carl.dong@windfindtech.com" created="Mon, 20 Jul 2015 07:04:54 +0000"  >&lt;p&gt;it is in __wt_tree_walk as you can see in screen shot new updated.&lt;/p&gt;

</comment>
                            <comment id="969677" author="michael.cahill" created="Mon, 20 Jul 2015 06:22:13 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=carl.dong%40windfindtech.com&quot; class=&quot;user-hover&quot; rel=&quot;carl.dong@windfindtech.com&quot;&gt;carl.dong@windfindtech.com&lt;/a&gt;, sorry you are having this problem with MongoDB.&lt;/p&gt;

&lt;p&gt;I have looked through the log, but unfortunately I cannot see the cause from there.  I don&apos;t think the message &quot;thread over memory limit, cleaning up&quot; is serious: it is just an indication that memory is being freed from a thread-local cache to be available to other threads.  That happens any time a mongod thread becomes idle with more than 64KB memory in its thread local cache.&lt;/p&gt;

&lt;p&gt;One way to be sure about what is happening would be to run &quot;perf top -g&quot; once mongod gets into this state and send us a screenshot of the busy functions.&lt;/p&gt;

&lt;p&gt;Without profiling information, my best guess is that you may be seeing poor behavior of the thread that truncates from the oplog to keep the size capped.  Tickets &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18875&quot; title=&quot;Oplog performance on WT degrades over time after accumulation of deleted items&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-18875&quot;&gt;&lt;del&gt;SERVER-18875&lt;/del&gt;&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-19178&quot; title=&quot;In WiredTiger capped collection truncates, avoid walking lists of deleted items&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-19178&quot;&gt;&lt;del&gt;SERVER-19178&lt;/del&gt;&lt;/a&gt; dealt with these problems: they will be fixed in the upcoming 3.0.5 release.&lt;/p&gt;</comment>
                            <comment id="969646" author="carl.dong@windfindtech.com" created="Mon, 20 Jul 2015 03:41:29 +0000"  >&lt;p&gt;Affect Version should be : 3.0.4 enterprise&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="212159">SERVER-19064</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="83498" name="2015-07-27.rar" size="1220574" author="carl.dong@windfindtech.com" created="Mon, 27 Jul 2015 03:13:12 +0000"/>
                            <attachment id="82399" name="CPU Screenshot.jpg" size="146358" author="carl.dong@windfindtech.com" created="Mon, 20 Jul 2015 03:35:47 +0000"/>
                            <attachment id="82400" name="DB.rar" size="1504159" author="carl.dong@windfindtech.com" created="Mon, 20 Jul 2015 03:35:47 +0000"/>
                            <attachment id="83639" name="DBlog0728.rar" size="1770415" author="carl.dong@windfindtech.com" created="Tue, 28 Jul 2015 02:51:07 +0000"/>
                            <attachment id="82397" name="Detail_Cpu_ScreenShot2.jpg" size="226126" author="carl.dong@windfindtech.com" created="Mon, 20 Jul 2015 03:35:47 +0000"/>
                            <attachment id="82398" name="Detail_Cpu_screenshot.jpg" size="190773" author="carl.dong@windfindtech.com" created="Mon, 20 Jul 2015 03:35:47 +0000"/>
                            <attachment id="82396" name="OpsManager_Screenshot.jpg" size="17104" author="carl.dong@windfindtech.com" created="Mon, 20 Jul 2015 03:35:47 +0000"/>
                            <attachment id="83544" name="cpu.png" size="207000" author="bruce.lucas@mongodb.com" created="Mon, 27 Jul 2015 13:38:51 +0000"/>
                            <attachment id="82528" name="cpu_usage.png" size="286926" author="lalo" created="Mon, 20 Jul 2015 16:48:36 +0000"/>
                            <attachment id="82529" name="llorenzen_serverStatus.json" size="15449" author="lalo" created="Mon, 20 Jul 2015 16:48:58 +0000"/>
                            <attachment id="83291" name="logs.rar" size="1385756" author="carl.dong@windfindtech.com" created="Fri, 24 Jul 2015 02:59:40 +0000"/>
                            <attachment id="82403" name="perf top sc.jpg" size="207822" author="carl.dong@windfindtech.com" created="Mon, 20 Jul 2015 07:04:41 +0000"/>
                            <attachment id="88164" name="serverStatus.txt" size="12982" author="huy" created="Mon, 31 Aug 2015 10:40:41 +0000"/>
                            <attachment id="82404" name="serverstatus.txt" size="27758" author="carl.dong@windfindtech.com" created="Mon, 20 Jul 2015 07:38:22 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>24.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 20 Jul 2015 06:22:13 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        8 years, 20 weeks, 4 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            8 years, 20 weeks, 4 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>bruce.lucas@mongodb.com</customfieldvalue>
            <customfieldvalue>carl.dong@windfindtech.com</customfieldvalue>
            <customfieldvalue>dan@mongodb.com</customfieldvalue>
            <customfieldvalue>huy</customfieldvalue>
            <customfieldvalue>lalo</customfieldvalue>
            <customfieldvalue>michael.cahill@mongodb.com</customfieldvalue>
            <customfieldvalue>mydigitalself</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrl05j:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hsbabj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hs14e7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>