<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:29:27 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-12733] Flush mmap files in parallel to achieve better flush times on Windows</title>
                <link>https://jira.mongodb.org/browse/SERVER-12733</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Presently mmap&apos;ed file flush happens in sequence for multiple files. This results in long flush times on Azure / Windows platform where the OS is not able to do concurrent flushes of the contents of the file. See &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-12401&quot; title=&quot;Improve the memory-mapped files flush on Windows&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-12401&quot;&gt;&lt;del&gt;SERVER-12401&lt;/del&gt;&lt;/a&gt; for more details.&lt;/p&gt;

&lt;p&gt;The issue is especially critical if there are a lot of random updates that dirty large parts of the mmap&apos;ed region. One way to do it, that is available in the short term and can be done solely on mongod side is to flush database files in parallel. We could see this does result in parallel data flush by OS and achieves higher throughput. This could be one of the ways to get better flush times on all platforms.&lt;/p&gt;

&lt;p&gt;The proposed changes would consist for number 2 are as follows:&lt;br/&gt;
1. A fixed number of threads (8 is proposed) using mongo::ThreadPool to process file flushes.&lt;br/&gt;
2. MongoFile::_flushAll will now schedule 1 file flush per file into this thread pool. When all flush requests are done, _flushAll will finish.&lt;br/&gt;
3. A change of _globalFlushMutex (a Windows only lock) to a Read Write Lock so that WRITETODATAFILES would take an exclusive lock, and file flushes would take a read lock. Individual flies are allowed to flush in parallel of each other per &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-7378&quot; title=&quot;Windows Azure logs errors and retries if WRITETODATAFILES and FlushViewOfFile run at the same time&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-7378&quot;&gt;&lt;del&gt;SERVER-7378&lt;/del&gt;&lt;/a&gt;, but not in parallel with WRITETODATAFILES. Also, we will ensure the lock is only held for the duration of the FlushViewOfFile call, and not the additional FlushFileBuffers call.&lt;/p&gt;</description>
                <environment></environment>
        <key id="111076">SERVER-12733</key>
            <summary>Flush mmap files in parallel to achieve better flush times on Windows</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="backlog-server-execution">Backlog - Storage Execution Team</assignee>
                                    <reporter username="anil.kumar">Anil Kumar</reporter>
                        <labels>
                            <label>Windows</label>
                    </labels>
                <created>Fri, 14 Feb 2014 16:45:33 +0000</created>
                <updated>Tue, 6 Dec 2022 05:11:42 +0000</updated>
                            <resolved>Fri, 14 Sep 2018 20:05:04 +0000</resolved>
                                                                    <component>MMAPv1</component>
                    <component>Performance</component>
                    <component>Storage</component>
                                        <votes>1</votes>
                                    <watches>10</watches>
                                                                                                                <comments>
                            <comment id="521976" author="schwerin" created="Mon, 24 Mar 2014 18:32:31 +0000"  >&lt;p&gt;On systems supporting the Posix mmap interface, you might be able to get away with one thread that msyncs all the files two times, once with the MS_ASYNC flag set, and once with the MS_SYNC flag set.  The first pass would let the OS start scheduling all of the disk i/o, while the second would essentially provide a barrier to wait for it to finish.  Maybe there&apos;s a better way to handle that barrier, too.&lt;/p&gt;</comment>
                            <comment id="499429" author="alex.komyagin@10gen.com" created="Fri, 14 Feb 2014 16:53:46 +0000"  >&lt;p&gt;i won&apos;t expect it to make any significant difference on platforms that are already doing async disk IO under the hood (e.g. linux), but should help for windows according to our tests&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25136"><![CDATA[Storage Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 14 Feb 2014 16:53:46 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        9 years, 47 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            9 years, 47 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>alex.komyagin@mongodb.com</customfieldvalue>
            <customfieldvalue>schwerin@mongodb.com</customfieldvalue>
            <customfieldvalue>anil.kumar</customfieldvalue>
            <customfieldvalue>backlog-server-execution</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrm1jz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr8hif:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>101661</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsbwd3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>