<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:22:45 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-30120] InMemory engine: &apos;inMemorySizeGB&apos; doesn&apos;t limit the process memory usage</title>
                <link>https://jira.mongodb.org/browse/SERVER-30120</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Using MongoDB Enterprise v3.2.15 on Ubuntu 16.04&lt;br/&gt;
Starting mongod as a service using:&lt;br/&gt;
/usr/bin/mongod --storageEngine inMemory --inMemorySizeGB 2 --quiet --config /etc/mongod.conf&lt;/p&gt;

&lt;p&gt;It seems that the memory limit doesn&apos;t really limit the process memory, mongod process could take more than 3GB of memory.&lt;/p&gt;

&lt;p&gt;Note that on our implementation we work with few hundreds of DBs.&lt;br/&gt;
Specifically our auto tests creates a large amount of databases and drop them after the test ends. However the memory isn&apos;t released.&lt;/p&gt;

&lt;p&gt;Also after a while the process start to take ~100% CPU even without any client load.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Assaf&lt;/p&gt;</description>
                <environment></environment>
        <key id="403414">SERVER-30120</key>
            <summary>InMemory engine: &apos;inMemorySizeGB&apos; doesn&apos;t limit the process memory usage</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13202">Works as Designed</resolution>
                                        <assignee username="mark.agarunov">Mark Agarunov</assignee>
                                    <reporter username="assaf@xmcyber.com">Assaf Oren</reporter>
                        <labels>
                    </labels>
                <created>Thu, 13 Jul 2017 06:57:15 +0000</created>
                <updated>Fri, 27 Oct 2023 14:31:43 +0000</updated>
                            <resolved>Fri, 27 Oct 2023 14:31:43 +0000</resolved>
                                    <version>3.2.15</version>
                                                    <component>Internal Code</component>
                    <component>Performance</component>
                    <component>Stability</component>
                                        <votes>0</votes>
                                    <watches>12</watches>
                                                                                                                <comments>
                            <comment id="1668762" author="assaf-oren" created="Sat, 9 Sep 2017 18:48:57 +0000"  >&lt;p&gt;Thanks for your response.&lt;br/&gt;
I&apos;ll keep track of issue &lt;a href=&quot;https://jira.mongodb.org/browse/WT-3566&quot; title=&quot;Handle memory is not reclaimed when using in-memory mode&quot; class=&quot;issue-link&quot; data-issue-key=&quot;WT-3566&quot;&gt;WT-3566&lt;/a&gt; since currently this cause our auto tests to stuck every few hours due to mongodb process taking all server&apos;s RAM.&lt;/p&gt;

&lt;p&gt;For the WiredTiger regular and in memory engines, isn&apos;t there any way to limit the amount of RAM taken by mongod process? (also for regular use without deleting DBs/collections)&lt;/p&gt;</comment>
                            <comment id="1666602" author="alexander.gorrod" created="Thu, 7 Sep 2017 07:15:55 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=assaf-oren&quot; class=&quot;user-hover&quot; rel=&quot;assaf-oren&quot;&gt;assaf-oren&lt;/a&gt; Sorry for the extended delay in getting a response to you on this ticket. First: we should have done a better job explaining that what you are reporting is generally expected behavior. The &lt;tt&gt;--inMemorySizeGB&lt;/tt&gt; option configures the buffer cache size for the storage engine - sorry that it is poorly named. There are many other components of MongoDB that use memory as well, for example:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Collection and index handle caches&lt;/li&gt;
	&lt;li&gt;Result caches for current operations&lt;/li&gt;
	&lt;li&gt;General data structure overhead&lt;/li&gt;
	&lt;li&gt;Temporary buffers used to optimize index builds, and some aggregation operations (these can consume between 100MB and 500MB each at a point in time).&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;The memory used outside the buffer cache varies widely depending on the workload - so you need to measure the usage for your application to understand how much memory is required.&lt;/p&gt;

&lt;p&gt;The reason we have been delaying answering is that there appears to also be a problem reclaiming space when a collection is dropped - so an application using the inMemory storage engine that is creating/dropping collections will observe increased memory usage over time. It is taking us time to isolate the root cause for this issue. I&apos;ve opened &lt;a href=&quot;https://jira.mongodb.org/browse/WT-3566&quot; title=&quot;Handle memory is not reclaimed when using in-memory mode&quot; class=&quot;issue-link&quot; data-issue-key=&quot;WT-3566&quot;&gt;WT-3566&lt;/a&gt; to further investigate that particular issue.&lt;/p&gt;

&lt;p&gt;The memory growth issue is specific to the inMemory storage engine. If the behavior you are seeing is similar when running the WiredTiger storage engine and the inMemory storage engine it is expected behavior, and you need to configure &lt;tt&gt;--inMemorySizeGB&lt;/tt&gt; to be small enough to allow enough RAM for the other memory used by MongoDB and other processes on the system.&lt;/p&gt;</comment>
                            <comment id="1664179" author="assaf-oren" created="Tue, 5 Sep 2017 09:19:04 +0000"  >&lt;p&gt;Hi Guys, any progress with this issue?&lt;br/&gt;
Do you have an estimation for a solution to be available?&lt;/p&gt;</comment>
                            <comment id="1651823" author="mark.agarunov" created="Fri, 18 Aug 2017 19:50:40 +0000"  >&lt;p&gt;Hello &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=assaf-oren&quot; class=&quot;user-hover&quot; rel=&quot;assaf-oren&quot;&gt;assaf-oren&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;My apologies for the delay in response. We are still investigating this behavior and will update you once more information becomes available. &lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Mark&lt;/p&gt;</comment>
                            <comment id="1639548" author="assaf-oren" created="Thu, 3 Aug 2017 11:47:12 +0000"  >&lt;p&gt;Hi,&lt;/p&gt;

&lt;p&gt;Is there any update on this?&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Assaf&lt;/p&gt;</comment>
                            <comment id="1632665" author="assaf-oren" created="Wed, 26 Jul 2017 19:37:15 +0000"  >&lt;p&gt;Hi Mark,&lt;/p&gt;

&lt;p&gt;I think there were about 4700 DBs created.&lt;br/&gt;
So the memory leak is on index allocations/releases? Index memory is never released?&lt;br/&gt;
Spacing these won&apos;t help, even when we delete all DBs and close all connections to the process, the memory won&apos;t go down.&lt;/p&gt;

&lt;p&gt;BTW, we tried the latest enterprise release - v3.4.6 and we can see this issue there as well.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Assaf&lt;/p&gt;</comment>
                            <comment id="1632571" author="mark.agarunov" created="Wed, 26 Jul 2017 18:29:23 +0000"  >&lt;p&gt;Hello &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=assaf-oren&quot; class=&quot;user-hover&quot; rel=&quot;assaf-oren&quot;&gt;assaf-oren&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thank you for providing the data and logs. After looking over this, it seems that the memory usage is mostly due to index builds. According to the logs, there were 199321 index builds during this duration. Spacing these out or reducing the number of index rebuilds may alleviate the memory usage.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Mark&lt;/p&gt;</comment>
                            <comment id="1630839" author="assaf-oren" created="Tue, 25 Jul 2017 06:03:47 +0000"  >&lt;p&gt;Hi,&lt;/p&gt;

&lt;p&gt;Is there any update on this?&lt;br/&gt;
Also, is there any way to limit the process memory? (not only the engine/cache size)&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Assaf&lt;/p&gt;</comment>
                            <comment id="1626109" author="assaf-oren" created="Wed, 19 Jul 2017 11:48:00 +0000"  >&lt;p&gt;Hi Mark,&lt;/p&gt;

&lt;p&gt;I uploaded two files (see those with &apos;2017_07_19&apos; in the filename), it include the diagnostics and full log.&lt;br/&gt;
The run from today is using --setParameter heapProfilingEnabled=true&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Assaf&lt;/p&gt;</comment>
                            <comment id="1623970" author="mark.agarunov" created="Mon, 17 Jul 2017 19:07:15 +0000"  >&lt;p&gt;Hello &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=assaf-oren&quot; class=&quot;user-hover&quot; rel=&quot;assaf-oren&quot;&gt;assaf-oren&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thank you for providing this data. After looking over this I agree that the memory is growing beyond what it should be using; this may be indicative of a memory leak. So that we can see exactly where the memory may be leaking, please do the following:&lt;/p&gt;

&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;Start mongod with the additional flag &lt;tt&gt;--setParameter heapProfilingEnabled=true&lt;/tt&gt;&lt;/li&gt;
	&lt;li&gt;Run mongod until reaching the high memory usage condition.&lt;/li&gt;
	&lt;li&gt;Archive and upload the $dbpath/diagnostic.data so that we can examine the data.&lt;/li&gt;
	&lt;li&gt;Provide the complete mongod logs while this issue is present.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Thanks,&lt;br/&gt;
Mark&lt;/p&gt;</comment>
                            <comment id="1623705" author="assaf-oren" created="Mon, 17 Jul 2017 16:01:38 +0000"  >&lt;p&gt;Thanks Mark, just upload the file &apos;JIRA-30120--....tar.gz&apos; including the diagnostic data for the  Jul 12 and 13 where mongo memory got above 4GB while limited to 2GB.&lt;/p&gt;</comment>
                            <comment id="1623647" author="mark.agarunov" created="Mon, 17 Jul 2017 15:37:33 +0000"  >&lt;p&gt;Hello &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=assaf-oren&quot; class=&quot;user-hover&quot; rel=&quot;assaf-oren&quot;&gt;assaf-oren&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;I&apos;ve generated a &lt;a href=&quot;https://10gen-httpsupload.s3.amazonaws.com/upload_forms/d71ed015-a593-4511-b7f7-d700a3f35856.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;secure upload portal&lt;/a&gt; so that you can send us the diagnostic data privately. The number of collections/indexes may cause overhead in some circumstances, however the diagnostic data may allow us to diagnose if this is the case.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Mark&lt;/p&gt;</comment>
                            <comment id="1622975" author="assaf-oren" created="Sun, 16 Jul 2017 06:20:51 +0000"  >&lt;p&gt;Thank you for your reply.&lt;br/&gt;
Is there a way to send you the diagnostic data in a private channel?&lt;/p&gt;

&lt;p&gt;I suspect the cause for this is the amount of DBs we are using.&lt;br/&gt;
Is there an overhead per DB (or collection/index) even if there is no client connected to this DB?&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Assaf&lt;/p&gt;</comment>
                            <comment id="1621655" author="mark.agarunov" created="Thu, 13 Jul 2017 21:45:44 +0000"  >&lt;p&gt;Hello &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=assaf-oren&quot; class=&quot;user-hover&quot; rel=&quot;assaf-oren&quot;&gt;assaf-oren&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thank you for the report. The &lt;tt&gt;--inMemorySizeGB&lt;/tt&gt; will set the size for the storage engine, not the entire process, so it is possible for the total memory usage to be greater than the value set for &lt;tt&gt;inMemorySizeGB&lt;/tt&gt;. To better investigate this, please archive and upload the &lt;tt&gt;$dbpath/diagnostic.data&lt;/tt&gt; directory so that we can get some insight into way may be causing the higher memory and high CPU usage.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Mark&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="424590">WT-3566</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="163576" name="IFfwJD.png" size="138704" author="mark.agarunov" created="Fri, 18 Aug 2017 19:41:39 +0000"/>
                            <attachment id="161340" name="stacks.png" size="81818" author="mark.agarunov" created="Wed, 19 Jul 2017 21:28:37 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>14.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 13 Jul 2017 21:45:44 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 22 weeks, 4 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 22 weeks, 4 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>alexander.gorrod@mongodb.com</customfieldvalue>
            <customfieldvalue>assaf@xmcyber.com</customfieldvalue>
            <customfieldvalue>assaf-oren</customfieldvalue>
            <customfieldvalue>mark.agarunov</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htat53:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|ht2pbb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;Start in memory mongod process using:&lt;br/&gt;
/usr/bin/mongod --storageEngine inMemory --inMemorySizeGB 2 --quiet --config /etc/mongod.conf&lt;/p&gt;

&lt;p&gt;Create enough DBs to make the process memory higher than 2GB.&lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htaf7r:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>