<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:34:18 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-14260] Reusing freed extents takes a long time</title>
                <link>https://jira.mongodb.org/browse/SERVER-14260</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;When we repeatedly create and drop large collections, the extent freelist gets very long and the server starts to take a long time to find a suitable freed extent to reuse while the database being write-locked. It can take from 10 sec to several minutes depending on the length of freelist. This is related to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3022&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;SERVER-3022&lt;/a&gt; if I understand it correctly.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;warning: newExtent 16365 scanned&lt;br/&gt;
warning: slow scan in allocFromFreeList (in write lock)&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;There would be three approaches to mitigate this problem:&lt;/p&gt;

&lt;ol&gt;
	&lt;li&gt;Let users specify the size of initial extent both for collection data and indices to possibly reduce the number of extents to be allocated. This used to be partially achieved by passing &lt;tt&gt;size&lt;/tt&gt; parameter on &lt;tt&gt;db.createCollection&lt;/tt&gt; but it&apos;s no longer possible since 2.6.0 (see &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-13144&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;SERVER-13144&lt;/a&gt;). If a collection is supposed to grow to a large size (say 100 MiB), it makes no sense to start from the minimum extent, namely 4096 bytes long.&lt;/li&gt;
	&lt;li&gt;Merge two freed extents if they are adjacent to each other. This could reduce the length of freelist.&lt;/li&gt;
	&lt;li&gt;Have a cache of extent freelist on heap for faster searching. It&apos;s so slow to do random access on mmapped pages due to repeated page faults and disk seek/reads.&lt;/li&gt;
&lt;/ol&gt;
</description>
                <environment></environment>
        <key id="142222">SERVER-14260</key>
            <summary>Reusing freed extents takes a long time</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="dan@mongodb.com">Daniel Pasette</assignee>
                                    <reporter username="pho">PHO</reporter>
                        <labels>
                    </labels>
                <created>Mon, 16 Jun 2014 09:17:54 +0000</created>
                <updated>Tue, 11 Jun 2019 15:32:21 +0000</updated>
                            <resolved>Fri, 22 May 2015 22:56:25 +0000</resolved>
                                                                    <component>Performance</component>
                    <component>Storage</component>
                                        <votes>1</votes>
                                    <watches>8</watches>
                                                                                                                <comments>
                            <comment id="871445" author="gregmurphy" created="Thu, 2 Apr 2015 15:10:14 +0000"  >&lt;p&gt;We&apos;re also experiencing this issue on 2.6.8. The database which is causing the problem has thousands of small collections created each day and then deleted a few days later. As a result, it has a huge extent free list. dbstats output for that database is:&lt;/p&gt;

&lt;p&gt;{&lt;br/&gt;
		&quot;collections&quot; : 34345,&lt;br/&gt;
		&quot;objects&quot; : 5053502,&lt;br/&gt;
		&quot;avgObjSize&quot; : 741.2781583939217,&lt;br/&gt;
		&quot;dataSize&quot; : NumberLong(&quot;3746050656&quot;),&lt;br/&gt;
		&quot;storageSize&quot; : NumberLong(&quot;6263025664&quot;),&lt;br/&gt;
		&quot;numExtents&quot; : 45764,&lt;br/&gt;
		&quot;indexes&quot; : 88726,&lt;br/&gt;
		&quot;indexSize&quot; : NumberLong(1558075792),&lt;br/&gt;
		&quot;fileSize&quot; : NumberLong(&quot;27837595648&quot;),&lt;br/&gt;
		&quot;nsSizeMB&quot; : 128,&lt;br/&gt;
		&quot;dataFileVersion&quot; : &lt;/p&gt;
{
			&quot;major&quot; : 4,
			&quot;minor&quot; : 5
		}
&lt;p&gt;,&lt;br/&gt;
		&quot;extentFreeList&quot; : &lt;/p&gt;
{
			&quot;num&quot; : 74073,
			&quot;totalSize&quot; : 8233771008
		}
&lt;p&gt;,&lt;br/&gt;
		&quot;ok&quot; : 1&lt;br/&gt;
	}&lt;/p&gt;

&lt;p&gt;Some example log entries from when the server hangs during extent free list searches are:&lt;/p&gt;

&lt;p&gt;warning: slow scan in allocFromFreeList (in write lock)&lt;br/&gt;
warning: newExtent 12462 scanned&lt;/p&gt;

&lt;p&gt;warning: slow scan in allocFromFreeList (in write lock)&lt;br/&gt;
warning: newExtent 21047 scanned&lt;/p&gt;

&lt;p&gt;warning: slow scan in allocFromFreeList (in write lock)&lt;br/&gt;
warning: newExtent 74075 scanned&lt;/p&gt;

&lt;p&gt;The server can be unresponsive for anything up to about 5 minutes during these scans.&lt;/p&gt;</comment>
                            <comment id="625829" author="pho" created="Wed, 18 Jun 2014 09:52:19 +0000"  >&lt;p&gt;Thank you for letting me know about &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-14082&quot; title=&quot;Excessive freelist scanning for MaxBucket&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-14082&quot;&gt;&lt;del&gt;SERVER-14082&lt;/del&gt;&lt;/a&gt;: it is itself informative to me. However, the problem I described above is about extent freelists, not record freelists that reside in each extent.&lt;/p&gt;

&lt;p&gt;The last time I saw the problem was when I was using MongoDB 2.4.1. But it has occurred on various versions and I think it still persists in the git HEAD from what I&apos;ve examined the codebase: &lt;tt&gt;MMAP1DatabaseCatalogEntry::createCollection&lt;/tt&gt; allocates an extent of 4096 bytes long, freed extents will never be merged, and &lt;tt&gt;MmapV1ExtentManager::_allocFromFreeList&lt;/tt&gt; walks through mmapped pages. If you still want me to try 2.6.2 then I will happily do so though.&lt;/p&gt;</comment>
                            <comment id="624940" author="ramon.fernandez" created="Tue, 17 Jun 2014 18:22:07 +0000"  >&lt;p&gt;What version of MongoDB are you using? If you&apos;re using 2.6.0 or 2.6.1 you may be running into &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-14082&quot; title=&quot;Excessive freelist scanning for MaxBucket&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-14082&quot;&gt;&lt;del&gt;SERVER-14082&lt;/del&gt;&lt;/a&gt;, which was fixed in 2.6.2. Is it possible for you to try out 2.6.2 and let us know if the problem persists?&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[5002K00000e6VKzQAM]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 17 Jun 2014 18:22:07 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        8 years, 45 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            8 years, 45 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>dan@mongodb.com</customfieldvalue>
            <customfieldvalue>gregmurphy</customfieldvalue>
            <customfieldvalue>pho</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlt9z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrzqbj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>122791</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hroq1b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>