<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:21:29 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-9802] Single-threaded journal compression becomes a bottleneck when using &quot;durable&quot; writes</title>
                <link>https://jira.mongodb.org/browse/SERVER-9802</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;When multiple threads save documents in &quot;durable&quot; fashion (I mean getLastError: 1, j: true) they all wait untill one thread running durThread() method persist these changes. Also this thread does compression, which becomes a bottleneck. More details are here &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9754?focusedCommentId=347003&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-347003&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-9754?focusedCommentId=347003&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-347003&lt;/a&gt;&lt;/p&gt;</description>
                <environment>Windows 7 Professional SP1 x64</environment>
        <key id="76969">SERVER-9802</key>
            <summary>Single-threaded journal compression becomes a bottleneck when using &quot;durable&quot; writes</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="backlog-server-execution">Backlog - Storage Execution Team</assignee>
                                    <reporter username="dnaumov">Dmitry Naumov</reporter>
                        <labels>
                    </labels>
                <created>Wed, 29 May 2013 07:51:01 +0000</created>
                <updated>Tue, 6 Dec 2022 05:20:37 +0000</updated>
                            <resolved>Fri, 14 Sep 2018 20:08:42 +0000</resolved>
                                    <version>2.4.3</version>
                                                    <component>MMAPv1</component>
                    <component>Performance</component>
                    <component>Storage</component>
                                        <votes>3</votes>
                                    <watches>10</watches>
                                                                                                                <comments>
                            <comment id="769966" author="mdcallag" created="Fri, 21 Nov 2014 14:56:33 +0000"  >&lt;p&gt;I don&apos;t think this is needed given the arrival of the engine API and WiredTiger in 2.8. I prefer that the core team at MongoDB focus on other features.&lt;/p&gt;</comment>
                            <comment id="769905" author="dnaumov" created="Fri, 21 Nov 2014 13:10:12 +0000"  >&lt;p&gt;After disabling journal compression is extracted to separate item, now we can discuss only the case when journalling is enabled and compression is turned on.&lt;/p&gt;

&lt;p&gt;Let me rephrase title of this issue: does it really makes sense to compress buffers in one thread while other writers are waiting this compression to finish? As far as I remember each buffer is compressed individually, so why don&apos;t let writer thread compress it before putting in flushing queue? In this case journalling thread responsibility narrows to flush these already compressed buffers to disk and release waiting writers.&lt;/p&gt;

&lt;p&gt;In other words - in this case we can leverage multi-core CPU.&lt;/p&gt;</comment>
                            <comment id="542259" author="schwerin" created="Wed, 9 Apr 2014 15:04:38 +0000"  >&lt;p&gt;I&apos;ve scheduled &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-13343&quot; title=&quot;Don&amp;#39;t compress journal writes that are less than 8k (or 4k)&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-13343&quot;&gt;&lt;del&gt;SERVER-13343&lt;/del&gt;&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-13344&quot; title=&quot;Pad journal writes to 4k not 8k&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-13344&quot;&gt;&lt;del&gt;SERVER-13344&lt;/del&gt;&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-13345&quot; title=&quot;Provide an option to disable journal compression&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-13345&quot;&gt;&lt;del&gt;SERVER-13345&lt;/del&gt;&lt;/a&gt; to the tail half of the 2.7/2.8 dev cycle, time permitting.  This should alleviate this problem.  Given that journal writing will always be one of the activities that can delay responding to durable writes, I&apos;m not entirely certain what the criteria for resolving this particular ticket can be.  Until we agree on the success criteria, I&apos;m going to mark this &quot;Needs Further Definition&quot;.  It&apos;s not a perfect categorization, but it&apos;s more accurate than &quot;Needs Triage&quot;.&lt;/p&gt;</comment>
                            <comment id="523156" author="dnaumov" created="Tue, 25 Mar 2014 20:39:16 +0000"  >&lt;p&gt;Mark,&lt;br/&gt;
I think disabling journal compression is a good thing to start. At least it should start scaling on cpu cores.&lt;/p&gt;</comment>
                            <comment id="522981" author="mdcallag" created="Tue, 25 Mar 2014 17:29:40 +0000"  >&lt;p&gt;Forcing journal file preallocation and recycling even for flash storage can help &amp;#8211; &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-13346&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-13346&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="522978" author="mdcallag" created="Tue, 25 Mar 2014 17:25:17 +0000"  >&lt;p&gt;Created &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-13345&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-13345&lt;/a&gt; asking for an option to disable journal compression. I have workloads that get better throughput with it disabled.&lt;/p&gt;</comment>
                            <comment id="522975" author="mdcallag" created="Tue, 25 Mar 2014 17:20:59 +0000"  >&lt;p&gt;Current code pads journal writes to a multiple of 8kb. For j:1 workloads with small changes padding to 4kb will help . See &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-13344&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-13344&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="522969" author="mdcallag" created="Tue, 25 Mar 2014 17:06:55 +0000"  >&lt;p&gt;I don&apos;t think the code should try to compress journal writes that are already less than 8kb &amp;#8211; &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-13343&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-13343&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="356686" author="dnaumov" created="Mon, 10 Jun 2013 10:05:23 +0000"  >&lt;p&gt;Should I provide some benchmarks or something? This issue was hidden by &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9754&quot; title=&quot;Every write to journal costs additional seek to update file metadata&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9754&quot;&gt;&lt;del&gt;SERVER-9754&lt;/del&gt;&lt;/a&gt;, but now it&apos;s negative effect should be more visible.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="127516">SERVER-13343</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="127521">SERVER-13344</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="127523">SERVER-13345</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25136"><![CDATA[Storage Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 25 Mar 2014 17:06:55 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        9 years, 12 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            9 years, 12 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>schwerin@mongodb.com</customfieldvalue>
            <customfieldvalue>backlog-server-execution</customfieldvalue>
            <customfieldvalue>dnaumov</customfieldvalue>
            <customfieldvalue>mdcallag</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrmrpz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr8hjz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7151</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrli4v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>