<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:32:57 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-33296] Excessive memory usage due to heap fragmentation</title>
                <link>https://jira.mongodb.org/browse/SERVER-33296</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;The changes described in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-20306&quot; title=&quot;75% excess memory usage under WiredTiger during stress test&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-20306&quot;&gt;&lt;del&gt;SERVER-20306&lt;/del&gt;&lt;/a&gt; eliminated a common source of memory fragmentation, but it can still occur for other reasons. Here&apos;s an example from a node undergoing initial sync:&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/179089/179089_fragmentation.png&quot; width=&quot;100%&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;Over time&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;allocated memory never exceeds 8 GB&lt;/li&gt;
	&lt;li&gt;but heap size and resident memory reach nearly 14 GB&lt;/li&gt;
	&lt;li&gt;this is due to an accumulation of pageheap_free_bytes&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;A common cause of this is a shifting distribution of allocated memory sizes, which leaves free pages dedicated to one size of buffer unable to be used for new memory requests because they are for a different size buffer.&lt;/p&gt;

&lt;p&gt;Setting TCMALLOC_AGGRESSIVE_DECOMMIT can address this issue by causing tcmalloc to aggressively return the free pages to the o/s where they can then be re-used by tcmalloc to satisfy new memory requests. However can have an unacceptable negative performance impact. Is there a tweak to tcmalloc that can give us better behavior for workloads like this?&lt;/p&gt;</description>
                <environment></environment>
        <key id="497374">SERVER-33296</key>
            <summary>Excessive memory usage due to heap fragmentation</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="ritesh.saigal@mongodb.com">Ritesh Saigal</assignee>
                                    <reporter username="bruce.lucas@mongodb.com">Bruce Lucas</reporter>
                        <labels>
                            <label>malloc</label>
                            <label>memory-management</label>
                            <label>perf-effort-xlarge</label>
                            <label>perf-improve-product</label>
                            <label>perf-urgency-asap</label>
                            <label>perf-value-essential</label>
                            <label>tcmalloc</label>
                    </labels>
                <created>Tue, 13 Feb 2018 18:49:44 +0000</created>
                <updated>Thu, 25 Jan 2024 20:37:45 +0000</updated>
                                            <version>3.6.2</version>
                                                    <component>WiredTiger</component>
                                        <votes>8</votes>
                                    <watches>59</watches>
                                                                                                                <comments>
                            <comment id="4238156" author="JIRAUSER1263881" created="Thu, 9 Dec 2021 06:40:40 +0000"  >&lt;p&gt;Hi,&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;My name is Eran Davidi and my company is a customer of MongoDB atlas.&#160;&lt;/p&gt;

&lt;p&gt;I wanted to ask if there is an estimation on when this bug will be fixed since I see it is very old.&lt;/p&gt;

&lt;p&gt;We are using Mongo 4.4.10 and suffering from this issue.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Best Regards,&lt;/p&gt;

&lt;p&gt;Eran Davidi&lt;/p&gt;</comment>
                            <comment id="4023956" author="JIRAUSER1256988" created="Thu, 26 Aug 2021 13:36:44 +0000"  >&lt;p&gt;I think the ultimate solution should be to migrate to the new per-cpu TCmalloc. I commented why I think it will be faster in the comments for PERF-2106 &quot;Performance support for TCMalloc Evaluation&quot;. Using other malloc solutions should be considered as well, however new TCmalloc might still be the best.&lt;/p&gt;</comment>
                            <comment id="1836480" author="ian@10gen.com" created="Fri, 16 Mar 2018 14:25:03 +0000"  >&lt;p&gt;Reviewed this in needs triage but the high cost:benefit puts this outside of the Storage team&apos;s top 15 at the moment.  So putting on backlog and will re-appraise later.&lt;/p&gt;

&lt;p&gt;CC &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=asya&quot; class=&quot;user-hover&quot; rel=&quot;asya&quot;&gt;asya&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1805452" author="bruce.lucas@10gen.com" created="Wed, 14 Feb 2018 14:24:14 +0000"  >&lt;p&gt;Verified that the same behavior is still observed in 3.6.2:&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/179152/179152_fragmentation-3.6.2.png&quot; width=&quot;100%&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                                        </outwardlinks>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="616475">SERVER-37541</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="681477">SERVER-39325</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="545602">SERVER-35046</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="441260">SERVER-31417</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="245033" name="4.3.3-rr100.png" size="337461" author="henrik.ingo@mongodb.com" created="Fri, 31 Jan 2020 10:46:15 +0000"/>
                            <attachment id="179152" name="fragmentation-3.6.2.png" size="109290" author="bruce.lucas@mongodb.com" created="Wed, 14 Feb 2018 14:23:29 +0000"/>
                            <attachment id="179089" name="fragmentation.png" size="106464" author="bruce.lucas@mongodb.com" created="Tue, 13 Feb 2018 18:56:08 +0000"/>
                            <attachment id="234347" name="image-2019-10-21-15-09-33-867.png" size="230464" author="henrik.ingo@mongodb.com" created="Mon, 21 Oct 2019 12:09:36 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25124"><![CDATA[Product Performance]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[500A000000ar1mwIAA, 5002K00000e94c9QAA, 5002K00000ei0ZpQAI, 5002K00000nD3NCQA0, 5002K00000obqNEQAY, 5002K00000ochpSQAQ, 5002K00000pnghXQAQ, 5002K00000pCWc7QAG, 5002K00000qded9QAA, 5002K00000qdwaxQAA, 5002K00000qfMMHQA2, 5002K00000r3B63QAE, 5002K00000rye0rQAA, 5002K00000s11OtQAI, 5002K00000s1bYVQAY, 5002K00000s2Kd4QAE, 5002K00000sbW2iQAE, 5002K00000scZXcQAM, 5002K00000sdUPpQAM, 5002K00000tUZIcQAO, 5002K00000tUrPFQA0, 5002K00000tVaySQAS, 5002K00000tRjLOQA0, 5002K00000tS7LgQAK, 5002K00000tS74aQAC, 5002K00000tR7i9QAC, 5002K00000uET1eQAG, 5002K00000vercVQAQ, 5002K00000vfd07QAA, 5002K00000vczpkQAA, 5002K000011gW5PQAU, 5002K000011gSwYQAU, 5002K000011ffKIQAY, 5002K0000131y5RQAQ, 5006R00001lAEuoQAG, 5006R00001hKsF2QAK, 5006R00001lRSwjQAG, 5006R00001nJS9fQAG, 5006R00001ntMdNQAU, 5006R00001oFU7ZQAW, 5006R00001owZDSQA2, 5006R00001plJP5QAM, 5006R00001owM1PQAU, 5006R00001plPIHQA2, 5006R00001pm90EQAQ, 5006R00001pmhLFQAY, 5006R00001pKdVAQA0, 5006R00001qGSAEQA4, 5006R00001qGcLpQAK, 5006R00001pnMxUQAU, 5006R00001qlROuQAM, 5006R00001qFtYLQA0, 5006R00001r6XJ9QAM, 5006R00001ra62eQAA, 5006R00001rakxRQAQ, 5006R00001raImcQAE, 5006R00001r65i7QAA, 5006R00001s7Q8UQAU, 5006R00001sCSShQAO, 5006R00001r7aRSQAY, 5006R00001srAnXQAU, 5006R00001s8CjxQAE, 5006R00001sttwGQAQ, 5006R00001tJutBQAS, 5006R00001tN1bAQAS, 5006R00001tNALAQA4, 5006R00001vxcxvQAA, 5006R00001tVLIbQAO, 5006R00001xjDHBQA2, 5006R00001y5ZVEQA2, 5006R00001yIR8uQAG]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 13 Feb 2018 19:02:47 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 8 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/PERF-1846'>PERF-1846</a></s>, <s><a href='https://jira.mongodb.org/browse/PM-2029'>PM-2029</a></s>, <a href='https://jira.mongodb.org/browse/PM-1812'>PM-1812</a>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ritesh.saigal@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 8 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10020"><![CDATA[Linux]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>andrew.shuvalov@mongodb.com</customfieldvalue>
            <customfieldvalue>bruce.lucas@mongodb.com</customfieldvalue>
            <customfieldvalue>eran@optibus.com</customfieldvalue>
            <customfieldvalue>ian@mongodb.com</customfieldvalue>
            <customfieldvalue>ritesh.saigal@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htqasn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hyaax3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2910">Dev Tools 2019-05-06</customfieldvalue>
    <customfieldvalue id="2942">Dev Tools 2019-04-22</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=david.daly&quot; class=&quot;user-hover&quot; rel=&quot;david.daly&quot;&gt;david.daly&lt;/a&gt; - Handing over to you per e-mail discussion.&lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htpwxz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>