<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:20:00 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-9303] Collection Size Growing Very Quickly Despite Document Removal</title>
                <link>https://jira.mongodb.org/browse/SERVER-9303</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Steps to Reproduce Issue : &lt;/p&gt;

&lt;p&gt;1.) Have a fairly large collection, ~10 million records, avg doc size is 1.4 kb.&lt;br/&gt;
2.) Replace each document (all 10 million) frequently with a remove() -&amp;gt; insert() (This works faster than an update because each remove+insert can be batched)&lt;/p&gt;

&lt;p&gt;After some time, according to stats the StorageSize gets huge, even though the totalSize is relatively small. In my case the StorageSize is 17 times larger than the totalSize.&lt;/p&gt;

&lt;p&gt;A single shard would have a 5 Gb total size, but a 100 Gb Storage Size.&lt;/p&gt;

&lt;p&gt;Here is the stats() for the collection in question&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;http://pastebin.com/qsbdJMQV&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://pastebin.com/qsbdJMQV&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;RS4 and RS5 are relatively empty because they are new shards.&lt;/p&gt;</description>
                <environment>5x Mongo Shards w/&lt;br/&gt;
&lt;br/&gt;
Ubuntu 10.04.4 LTS&lt;br/&gt;
64 - bit&lt;br/&gt;
830GB Drives&lt;br/&gt;
Intel(R) Xeon(R) CPU E3-1220 V2 @ 3.10GHz&lt;br/&gt;
32GB Ram&lt;br/&gt;
</environment>
        <key id="71250">SERVER-9303</key>
            <summary>Collection Size Growing Very Quickly Despite Document Removal</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="4">Incomplete</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="nimiw">Nimi Wariboko Jr.</reporter>
                        <labels>
                            <label>bson</label>
                            <label>performance</label>
                    </labels>
                <created>Tue, 9 Apr 2013 17:20:16 +0000</created>
                <updated>Thu, 11 Apr 2013 14:55:45 +0000</updated>
                            <resolved>Thu, 11 Apr 2013 14:55:45 +0000</resolved>
                                    <version>2.2.3</version>
                    <version>2.2.4</version>
                                                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="309538" author="scotthernandez" created="Tue, 9 Apr 2013 17:51:01 +0000"  >&lt;p&gt;Yes, dropping collections is a fast and more efficient way to reuse space (internally). Doing it at the db level is also fine and will release space to the filesystem in addition. Many people have a rotating system where they fill collections by time periods (or some of the criteria) and drop/create them periodically.&lt;/p&gt;

&lt;p&gt;Dropping a collection is much faster than deleting documents as well. With a sharded collection it is best not to reuse the collection names if you aren&apos;t on 2.4.x (mostly because of inefficiencies we fixed in newer versions).&lt;/p&gt;</comment>
                            <comment id="309534" author="nimiw" created="Tue, 9 Apr 2013 17:43:32 +0000"  >&lt;p&gt;So is the issue that the new documents are unable to fit in the old space? I had assumed since each document was a fixed size that fragmentation wouldn&apos;t be too much of an issue. Anyways I have turned it on, and I&apos;ll monitor the size of the new shard.&lt;/p&gt;

&lt;p&gt;This probably isn&apos;t the place for this however, but could I simply drop the collection to reclaim the used space, or is a repair my only option?&lt;/p&gt;</comment>
                            <comment id="309523" author="scotthernandez" created="Tue, 9 Apr 2013 17:30:26 +0000"  >&lt;p&gt;You may be seeing poor reuse of space for the deleted documents. &lt;/p&gt;

&lt;p&gt;Can you enable power of 2 allocation with &lt;a href=&quot;http://docs.mongodb.org/manual/reference/command/collMod/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://docs.mongodb.org/manual/reference/command/collMod/&lt;/a&gt; which should make the space much more reusable after your deletes. If that works better you will want to repair each shard to compact your database. I would suggest turning this on for your new shards now as a test.&lt;/p&gt;

&lt;p&gt;The algorithm is also slightly better in 2.4.x so upgrading and using that option may be best if that help with your issue.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 9 Apr 2013 17:30:26 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        10 years, 45 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            10 years, 45 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>nimiw</customfieldvalue>
            <customfieldvalue>scotthernandez</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrmxlr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrn6ov:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>49025</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hs0skv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>