<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:09:36 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-5682] writes cause non-trivial read i/o</title>
                <link>https://jira.mongodb.org/browse/SERVER-5682</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I&apos;ve run into a problem with a mongodb setup (2.0.2) that is only written to yet I see significant read i/o that slows down the performance quite a bit. The setup is a simple 3-node mongodb cluster (primary, secondary, arbiter) where the primary and secondary nodes have 8gb of ram and run a replicated set.&lt;/p&gt;

&lt;p&gt;With a collection size of about 44gb I see a read-to-write ration of 3:1 that is there are actually more reads than writes happening on the system. After renaming the collection and starting a new one from scratch this ration changed to 1:4 which is significantly better but I still don&apos;t understand what causes the reads. The entire collection fits in memory right now so I&apos;m not sure what mongo could be reading from the disk all the time.&lt;br/&gt;
Once I stop writing to the collection the reads also disappear.&lt;/p&gt;</description>
                <environment></environment>
        <key id="36917">SERVER-5682</key>
            <summary>writes cause non-trivial read i/o</summary>
                <type id="6" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14720&amp;avatarType=issuetype">Question</type>
                                            <priority id="4" iconUrl="https://jira.mongodb.org/images/icons/priorities/minor.svg">Minor - P4</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="4">Incomplete</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="dennis">Dennis Jacobfeuerborn</reporter>
                        <labels>
                            <label>performance</label>
                    </labels>
                <created>Mon, 23 Apr 2012 01:22:20 +0000</created>
                <updated>Wed, 15 Aug 2012 14:24:21 +0000</updated>
                            <resolved>Tue, 10 Jul 2012 20:04:03 +0000</resolved>
                                    <version>2.0.2</version>
                                                    <component>Performance</component>
                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="141523" author="ian@10gen.com" created="Tue, 10 Jul 2012 20:04:03 +0000"  >&lt;p&gt;@dennis I&apos;m closing this for now, but please reopen if you see this again or if you can provide any further info.&lt;/p&gt;</comment>
                            <comment id="112553" author="scotthernandez" created="Mon, 23 Apr 2012 13:40:14 +0000"  >&lt;p&gt;Please provide the stats for your collection and database. If indexes don&apos;t fit in 8GB then it would very likely need to do many reads during inserts. Also, do you even update or remove documents?&lt;/p&gt;

&lt;p&gt;Please included those stats when this happens again.&lt;/p&gt;</comment>
                            <comment id="112549" author="dennis" created="Mon, 23 Apr 2012 13:23:38 +0000"  >&lt;p&gt;The nodes have 8gb of memory.&lt;/p&gt;

&lt;p&gt;I attached a graph of cpu utilization during the last month. You can see i/o wait grow until we &quot;reboot&quot; the collection. From then on everything looks good again.&lt;/p&gt;

&lt;p&gt;Since this is a production system there isn&apos;t much we can do to experiment but it seems as the collection outgrows the ram the i/o becomes increasingly problematic. That would be expected if we&apos;d do a lot of reads on a working set that doesn&apos;t fit into memory but for a write-only system these reads should not exist.&lt;/p&gt;

&lt;p&gt;At the moment I see 90kb read for 10MB written which looks much more in line with what I would expect. Theoretically it should stay this way even as the collection grows though.&lt;/p&gt;</comment>
                            <comment id="112424" author="scotthernandez" created="Mon, 23 Apr 2012 01:39:01 +0000"  >&lt;p&gt;Do you have a way of reproducing this behavior?&lt;/p&gt;

&lt;p&gt;Also, what does &quot;free -ltm&quot;, &quot;iostat -xtm 2&quot; and mongostat show during the time when you don&apos;t expect reads happening? &lt;/p&gt;

&lt;p&gt;If you have 44GB of data, how much memory do you have? &lt;/p&gt;

&lt;p&gt;For your collection how many indexes do you have? What does db.coll.stats() look like?&lt;/p&gt;</comment>
                            <comment id="112421" author="dennis" created="Mon, 23 Apr 2012 01:28:33 +0000"  >&lt;p&gt;Some additional info: This happens on a Centos system with the filesystem mounted using the relatime option and shutting down the secondary node doesn&apos;t change the behaviour of the I/O. &lt;br/&gt;
I measured the I/O using &quot;iotop -Pa&quot; to make sure that it is indeed the mongod process that is causing the I/O and not something else on the system even though this is a minimal Centos 6 install with only the mongo rpms installed. &lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="16131" name="mongo.png" size="34875" author="dennis" created="Mon, 23 Apr 2012 13:23:38 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 23 Apr 2012 01:39:01 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        11 years, 32 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            11 years, 32 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>dennis</customfieldvalue>
            <customfieldvalue>ian@mongodb.com</customfieldvalue>
            <customfieldvalue>scotthernandez</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hro50v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hripzj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>22863</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10166" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Tests Written</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10153"><![CDATA[Unneeded]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht07t3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>