<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 02:55:44 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-989] High heap usage with lots of collections</title>
                <link>https://jira.mongodb.org/browse/SERVER-989</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I ran into very high RSS memory usage (mstearn thought the heap stat looked quite high) when testing mongod with a high number of collections (80k with only the automatic _id index per collection).  Dropping some collections (~100) temporarily dropped heap usage, but it raised &lt;b&gt;higher&lt;/b&gt; than before once dropping script stopped.&lt;/p&gt;

&lt;p&gt;Steps to reproduce:&lt;/p&gt;

&lt;p&gt;Start mongod with:&lt;/p&gt;

&lt;p&gt;mongod --dbpath=tmpmongo --port=27018 -v --nssize=2047&lt;/p&gt;

&lt;p&gt;Run the following code (or similar in the language of your choice):&lt;/p&gt;

&lt;p&gt;import pymongo&lt;br/&gt;
conn = pymongo.Connection(port=27018)&lt;br/&gt;
db = conn.testdb&lt;/p&gt;

&lt;p&gt;for x in range(1000000): # I never let it run past ~88k&lt;br/&gt;
    foo = db&lt;span class=&quot;error&quot;&gt;&amp;#91;&amp;quot;col%d&amp;quot; % x&amp;#93;&lt;/span&gt;.insert(&lt;/p&gt;
{&apos;t&apos;:0}
&lt;p&gt;, safe=True)&lt;br/&gt;
    if x % 100 == 0: print &quot;%6d: %r&quot; % (x, foo) # Optional obviously&lt;/p&gt;

&lt;ol&gt;
	&lt;li&gt;EOF&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;Essentially the code inserts a single document containing &lt;/p&gt;
{t:0}
&lt;p&gt; into a series of new collections.&lt;/p&gt;

&lt;p&gt;I used the following code to remove some collections:&lt;/p&gt;

&lt;ol&gt;
	&lt;li&gt;Approximately 100 collections before where I killed it.&lt;br/&gt;
for x in range(87635, 1000000): &lt;br/&gt;
    db.drop_collection(db&lt;span class=&quot;error&quot;&gt;&amp;#91;&amp;quot;col%d&amp;quot; % x&amp;#93;&lt;/span&gt;)&lt;/li&gt;
&lt;/ol&gt;


&lt;ol&gt;
	&lt;li&gt;EOF&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;End stats:&lt;br/&gt;
Namespaces: 175273&lt;br/&gt;
heap_usage_bytes : 1701860208&lt;/p&gt;

&lt;p&gt;Stats at peak number of collections:&lt;br/&gt;
namespaces: 175509&lt;br/&gt;
heap_usage_bytes: 1702426528&lt;/p&gt;

&lt;p&gt;Heap directly after removal script ended:&lt;br/&gt;
1703393952&lt;/p&gt;

&lt;p&gt;I did do a few other actions on the server (creating another test db &amp;amp; collection), but hopefully nothing that would drastically affect the statistics.&lt;/p&gt;

&lt;p&gt;If this is normal, could the Lots of Collections wiki page simply be updated with some indication as to how much RSS memory namespaces will use?&lt;/p&gt;

&lt;p&gt;I&apos;m trying to evaluate using lots of collections vs. a large collection with a heavily used index.&lt;/p&gt;</description>
                <environment>Ubuntu 9.10 64bit</environment>
        <key id="11738">SERVER-989</key>
            <summary>High heap usage with lots of collections</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="5">Cannot Reproduce</resolution>
                                        <assignee username="eliot">Eliot Horowitz</assignee>
                                    <reporter username="schmichael">Michael Schurter</reporter>
                        <labels>
                    </labels>
                <created>Thu, 8 Apr 2010 20:13:21 +0000</created>
                <updated>Fri, 21 Oct 2011 07:54:17 +0000</updated>
                            <resolved>Fri, 21 Oct 2011 07:54:17 +0000</resolved>
                                    <version>1.4.0</version>
                                                    <component>Performance</component>
                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                        <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 20 Oct 2011 14:19:55 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        13 years, 45 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            13 years, 45 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>eliot</customfieldvalue>
            <customfieldvalue>schmichael</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrpnvb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hriopr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>22656</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht0e5b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>