<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 02:58:35 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-1966] Don&apos;t lock reads on background flush (MemoryMappedFile::flushAll)</title>
                <link>https://jira.mongodb.org/browse/SERVER-1966</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;It seems like when the periodic flush (syncdelay) happens that reads are slowed done (because of locking for the flush).&lt;/p&gt;</description>
                <environment></environment>
        <key id="13417">SERVER-1966</key>
            <summary>Don&apos;t lock reads on background flush (MemoryMappedFile::flushAll)</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="5">Cannot Reproduce</resolution>
                                        <assignee username="eliot">Eliot Horowitz</assignee>
                                    <reporter username="scotthernandez">Scott Hernandez</reporter>
                        <labels>
                    </labels>
                <created>Mon, 18 Oct 2010 21:33:54 +0000</created>
                <updated>Fri, 30 Mar 2012 14:37:16 +0000</updated>
                            <resolved>Sat, 10 Sep 2011 04:39:42 +0000</resolved>
                                                                    <component>Concurrency</component>
                                        <votes>5</votes>
                                    <watches>11</watches>
                                                                                                                <comments>
                            <comment id="53313" author="eliot" created="Sat, 10 Sep 2011 04:39:42 +0000"  >&lt;p&gt;can&apos;t reproduce - a test case would help&lt;/p&gt;</comment>
                            <comment id="19722" author="eliot" created="Wed, 27 Oct 2010 20:10:43 +0000"  >&lt;p&gt;Yes - once a write is blocking - then reads pile up behind it&lt;/p&gt;</comment>
                            <comment id="19691" author="bwaldvogel" created="Wed, 27 Oct 2010 08:35:59 +0000"  >&lt;p&gt;Okay, but isn&apos;t it true that reads are blocked once the writes are blocked?&lt;/p&gt;</comment>
                            <comment id="19685" author="eliot" created="Wed, 27 Oct 2010 04:45:51 +0000"  >&lt;p&gt;Not entirely that&apos;s what happened - but one thing to try is decreasing --syncdelay.&lt;br/&gt;
It could increase overall disk load - but may make things smoother.&lt;/p&gt;

&lt;p&gt;If the sync takes longer than snycdelay, it will just keep syncing (still 1 thread at a time)&lt;/p&gt;</comment>
                            <comment id="19658" author="bwaldvogel" created="Tue, 26 Oct 2010 17:16:23 +0000"  >&lt;p&gt;Actually I was the guy who spoke to Scott and mentioned the locked reads while a flush was happening.&lt;br/&gt;
In the meantime I invested some more time to investigate the problem. I found out that the background flush wasn&apos;t really reason, though it could be part of the problem.&lt;/p&gt;

&lt;p&gt;I&apos;m running a benchmark that inserts/updates items in a collection as fast as possible (roughly 5-10mb/s). At the same time a couple of threads regularly fetch documents and measure the latency.&lt;br/&gt;
I carefully monitor the machines. The machines have enough RAM to hold the entire collection in memory.&lt;/p&gt;

&lt;p&gt;My assumption was that the reads will never ever block (ie. have a low latency) since there&apos;s no swapping. But actually, this assumption was wrong. I had latency spikes with over 1s delay. My inserts and updates caused so many dirty pages that the OS started writing the blocks to the disk. However, the disks are slower than the inserts and updates so the total sum of dirty pages (see /proc/meminfo) exceeded the dirty_ratio limit. The OS then blocked writes and since they hold the rw locks, the reads were also blocked. Or is this implication wrong here?&lt;/p&gt;

&lt;p&gt;Is this a known issue? And what is the best way to avoid that problem? (rate-limiting the updates or setting dirty_ratio to 100%?)&lt;/p&gt;

&lt;p&gt;BTW: what happens if the background sync takes for instance 6s but the syncdelay is set to 5s?&lt;/p&gt;</comment>
                            <comment id="19323" author="eliot" created="Tue, 19 Oct 2010 04:48:03 +0000"  >&lt;p&gt;This should not happen.&lt;br/&gt;
The flushes are totally in the background.&lt;br/&gt;
If you&apos;ve seen - please send log files to correspond.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                                        </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 19 Oct 2010 04:48:03 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        12 years, 23 weeks, 4 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SUPPORT-137'>SUPPORT-137</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            12 years, 23 weeks, 4 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>bwaldvogel</customfieldvalue>
            <customfieldvalue>eliot</customfieldvalue>
            <customfieldvalue>scotthernandez</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrpcw7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrigk7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>21335</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht0cbb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>