<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:47:03 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-18241] MongoDB 3.0 + WiredTiger process uses more cache memory than specified</title>
                <link>https://jira.mongodb.org/browse/SERVER-18241</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I&apos;ve migrated my write-heavy application to MongoDB 3 and WiredTiger. This helped a lot with some of the performance issues I had, but now I am experiencing some memory problems on my DB host. My setup is fairly simple: I have an EC2 instance with 30GB system memory dedicated to running MongoDB; I only have one mongod process; I don&apos;t use sharding or replica sets.&lt;/p&gt;

&lt;p&gt;When I first started using v3 and WiredTiger I&apos;ve set the cache size to something large, like 27GB. Considering that I&apos;m not running anything else on the instance, I thought it was safe to leave 3GB for the linux kernel and non-cache related MongoDB objects. My mongod was killed by OOM event on the next day. Then again after another day.&lt;/p&gt;

&lt;p&gt;I&apos;ve changed the cacheSizeGB to 24, leaving 6GB for the linux kernel and non-cache related MongoDB objects. Many days later I am running out of memory again.&lt;/p&gt;

&lt;p&gt;I&apos;ve been running the same application on Mongo 2.x for months without restarting the DB process. I don&apos;t use noTimeout cursors. I only have 8 web worker processes and 10 delayed_job processes accessing the database. That puts a cap of about 30-40 cursors that I might have open at the same time. It&apos;s far fewer most of the time.&lt;/p&gt;

&lt;p&gt;I&apos;m not sure how I can help you debug. `top` reports 26g memory used by the mongod process or 28.8g virtual image size. That&apos;s far above what is allowed for cache. Attached is the `top -abcn1` output and serverStatus info.&lt;/p&gt;

&lt;p&gt;Possibly related tickets: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-17456&quot; title=&quot;Mongodb 3.0 wiredTiger storage engine memory usage too high.&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-17456&quot;&gt;&lt;del&gt;SERVER-17456&lt;/del&gt;&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-17424&quot; title=&quot;WiredTiger uses substantially more memory than accounted for by cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-17424&quot;&gt;&lt;del&gt;SERVER-17424&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</description>
                <environment></environment>
        <key id="200798">SERVER-18241</key>
            <summary>MongoDB 3.0 + WiredTiger process uses more cache memory than specified</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="ramon.fernandez@mongodb.com">Ramon Fernandez Marina</assignee>
                                    <reporter username="amw">Adam Wr&#243;bel</reporter>
                        <labels>
                    </labels>
                <created>Tue, 28 Apr 2015 16:42:39 +0000</created>
                <updated>Tue, 28 Apr 2015 18:33:11 +0000</updated>
                            <resolved>Tue, 28 Apr 2015 17:59:24 +0000</resolved>
                                    <version>3.0.2</version>
                                                    <component>Stability</component>
                    <component>WiredTiger</component>
                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="900104" author="amw" created="Tue, 28 Apr 2015 18:33:11 +0000"  >&lt;p&gt;Ok, I will change the cache to default (15G) and we&apos;ll see if the process can outgrow it by another 10G. I will let you know if it does. It will likely take a long time though. Accumulating 3G over the allowed size took many days.&lt;/p&gt;

&lt;p&gt;In the attached serverStatus.json &quot;bytes currently in the cache&quot; is much lower than &quot;maximum bytes configured&quot; so it would suggest WiredTiger thinks it isn&apos;t even using all of the available cache.&lt;/p&gt;

&lt;p&gt;I thought WT internal cache was supposed to replace system-level file caching. I think there was even a mention of possibly moving to a raw disk layout in the future (space not controlled by file system). Seems kind of redundant to keep the same information cached in memory twice &#8211; once on the application level by WT and then again as system cache of the underlaying data file. Reads should be served by WT cache or by disk access while writes can be buffered by system, but only for very small chunks. Is that not the case? I don&apos;t have any other processes running on the instance that would require a lot of file system cache.&lt;/p&gt;

&lt;p&gt;AMI EC2 instances by default come without swap configured and this one is no exception. I find it a very efficient setup for applications which memory usage I can predict. OOM events are results of bugs either in my application or higher in the stack and should either be fixed or accommodated for. Swap leads to unreliable performance and doesn&apos;t remove the need to restart the leaking process, only slightly delays it. In either case system memory usage needs to be monitored.&lt;/p&gt;</comment>
                            <comment id="900057" author="ramon.fernandez" created="Tue, 28 Apr 2015 17:58:48 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=amw&quot; class=&quot;user-hover&quot; rel=&quot;amw&quot;&gt;amw&lt;/a&gt;, as you point out the symptoms are similar to those of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-17424&quot; title=&quot;WiredTiger uses substantially more memory than accounted for by cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-17424&quot;&gt;&lt;del&gt;SERVER-17424&lt;/del&gt;&lt;/a&gt;. While we don&apos;t have evidence of a leak yet, it is the case that the cache usage may grow higher than specified.&lt;/p&gt;

&lt;p&gt;I&apos;d recommend that you use the default cache setting (which will be half of available RAM) and monitor your system to see how much more over the limit is used. The goal is to find out if the used cache size grows without bound (which could indicate a bug or a leak) or if it grows above the limit but to a steady number (just a bug).&lt;/p&gt;

&lt;p&gt;Note that your current setting may starve the linux kernel from memory for file cache, which will negatively impact performance, so lowering the cache size may actually be advantageous for your write-heavy application.&lt;/p&gt;

&lt;p&gt;The other important aspect is the amount of swap available in your system. Having low or no swap will prevent the system from degrading gracefully and force the OOM killer to step in.&lt;/p&gt;

&lt;p&gt;I am going to close this ticket as a duplicate of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-17424&quot; title=&quot;WiredTiger uses substantially more memory than accounted for by cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-17424&quot;&gt;&lt;del&gt;SERVER-17424&lt;/del&gt;&lt;/a&gt;, but if you have some additional information that&apos;s specific to this ticket please log it here. If this information reveals that these two tickets have a different cause we&apos;ll reopen this one and continue the investigation.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="186885">SERVER-17424</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="72190" name="serverStatus.json" size="11755" author="amw" created="Tue, 28 Apr 2015 16:42:39 +0000"/>
                            <attachment id="72191" name="top_memory.txt" size="807" author="amw" created="Tue, 28 Apr 2015 16:42:39 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 28 Apr 2015 16:45:37 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        8 years, 42 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>amw</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            8 years, 42 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10020"><![CDATA[Linux]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>amw</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrl787:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hs8u9j:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsg2mv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>