<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:33:44 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-33537] High write IO after 3.6 upgrade</title>
                <link>https://jira.mongodb.org/browse/SERVER-33537</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Yesterday I upgraded one of our 3.4 mongodb cluster to 3.6 (3.4.9 -&amp;gt; 3.6.3).&lt;br/&gt;
After upgrade I noticed increased write IO (IOPS and throughput).&lt;br/&gt;
Added also 2 screenshots, upgrade was done at 5:20.&lt;/p&gt;

&lt;p&gt;When looking with iotop, I can see WTOplog.lThread is writing a lot, does it mean that 3.6 writes more oplog? Is it something that is expected or is it regression?&lt;/p&gt;

</description>
                <environment>hw: AWS c4.2xlarge&lt;br/&gt;
operating system: Ubuntu xenial&lt;br/&gt;
kernel: 4.4.0-92-generic #115-Ubuntu SMP Thu Aug 10 09:04:33 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux &lt;br/&gt;
mongodb volume: 130GiB io1 1000 IOPS&lt;br/&gt;
&lt;br/&gt;
conf:&lt;br/&gt;
storage:&lt;br/&gt;
&amp;nbsp;&amp;nbsp;dbPath: /data/data&lt;br/&gt;
&amp;nbsp;&amp;nbsp;journal:&lt;br/&gt;
&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;enabled: true&lt;br/&gt;
&amp;nbsp;&amp;nbsp;engine: wiredTiger&lt;br/&gt;
&lt;br/&gt;
systemLog:&lt;br/&gt;
&amp;nbsp;&amp;nbsp;destination: file&lt;br/&gt;
&amp;nbsp;&amp;nbsp;logAppend: true&lt;br/&gt;
&amp;nbsp;&amp;nbsp;path: /data/log/mongod.log&lt;br/&gt;
&lt;br/&gt;
net:&lt;br/&gt;
&amp;nbsp;&amp;nbsp;port: 27017&lt;br/&gt;
&amp;nbsp;&amp;nbsp;bindIp: 0.0.0.0&lt;br/&gt;
&lt;br/&gt;
replication:&lt;br/&gt;
&amp;nbsp;&amp;nbsp;oplogSizeMB: 512&lt;br/&gt;
&amp;nbsp;&amp;nbsp;replSetName: foo&lt;br/&gt;
&lt;br/&gt;
operationProfiling:&lt;br/&gt;
&amp;nbsp;&amp;nbsp;slowOpThresholdMs: 1&lt;br/&gt;
&amp;nbsp;&amp;nbsp;mode: off</environment>
        <key id="503419">SERVER-33537</key>
            <summary>High write IO after 3.6 upgrade</summary>
                <type id="6" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14720&amp;avatarType=issuetype">Question</type>
                                            <priority id="4" iconUrl="https://jira.mongodb.org/images/icons/priorities/minor.svg">Minor - P4</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="tarvip">Tarvi Pillessaar</reporter>
                        <labels>
                            <label>SWKB</label>
                    </labels>
                <created>Wed, 28 Feb 2018 08:21:01 +0000</created>
                <updated>Wed, 28 Feb 2018 15:10:32 +0000</updated>
                            <resolved>Wed, 28 Feb 2018 14:51:41 +0000</resolved>
                                    <version>3.6.3</version>
                                                    <component>Write Ops</component>
                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="1818510" author="bruce.lucas@10gen.com" created="Wed, 28 Feb 2018 14:51:02 +0000"  >&lt;p&gt;Hi Tarvi,&lt;/p&gt;

&lt;p&gt;Thanks for the data. Here&apos;s what we can see:&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/180496/180496_flushes.png&quot; width=&quot;100%&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;The rate of write operations before and after the upgrade is the same, but after the upgrade we see a much higher rate of log (journal) flushes. This is symptomatic of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-31679&quot; title=&quot;Increase in disk i/o for writes to replica set&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-31679&quot;&gt;&lt;del&gt;SERVER-31679&lt;/del&gt;&lt;/a&gt;, and typically affects workloads with a low degree of write concurrency, which appears to be the case here.&lt;/p&gt;

&lt;p&gt;Thanks for your report. A fix for this is in progress and will be in a future version of 3.6. I&apos;ll close this ticket as a duplicate of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-31679&quot; title=&quot;Increase in disk i/o for writes to replica set&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-31679&quot;&gt;&lt;del&gt;SERVER-31679&lt;/del&gt;&lt;/a&gt;; please watch that ticket for further updates.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Bruce&lt;/p&gt;</comment>
                            <comment id="1818483" author="tarvip" created="Wed, 28 Feb 2018 14:31:12 +0000"  >&lt;p&gt;Hi Bruce,&lt;/p&gt;

&lt;p&gt;Thanks you for quick response. Uploaded diagnostic.data, removed older metrics files to reduce archive size, hopefully it is okay.&lt;/p&gt;

&lt;p&gt;Tarvi&lt;/p&gt;</comment>
                            <comment id="1818443" author="bruce.lucas@10gen.com" created="Wed, 28 Feb 2018 14:01:34 +0000"  >&lt;p&gt;Hi Tarvi,&lt;/p&gt;

&lt;p&gt;It is possible that you&apos;re encountering &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-31679&quot; title=&quot;Increase in disk i/o for writes to replica set&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-31679&quot;&gt;&lt;del&gt;SERVER-31679&lt;/del&gt;&lt;/a&gt;. Can you please archive and upload the $dbpath/diagnostic.data directory from an affected node so we can confirm?&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Bruce&lt;/p&gt;</comment>
                            <comment id="1818256" author="tarvip" created="Wed, 28 Feb 2018 08:27:13 +0000"  >&lt;p&gt;Added 2 additional screenshots with longer timeframe. &lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="449479">SERVER-31679</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="180467" name="Screen Shot 2018-02-28 at 10.03.46.png" size="830230" author="tarvip" created="Wed, 28 Feb 2018 08:10:32 +0000"/>
                            <attachment id="180466" name="Screen Shot 2018-02-28 at 10.07.47.png" size="788923" author="tarvip" created="Wed, 28 Feb 2018 08:10:32 +0000"/>
                            <attachment id="180468" name="Screen Shot 2018-02-28 at 10.25.08.png" size="875887" author="tarvip" created="Wed, 28 Feb 2018 08:25:58 +0000"/>
                            <attachment id="180469" name="Screen Shot 2018-02-28 at 10.25.32.png" size="863013" author="tarvip" created="Wed, 28 Feb 2018 08:25:59 +0000"/>
                            <attachment id="180494" name="diagnostic.data.tar" size="65454080" author="tarvip" created="Wed, 28 Feb 2018 14:29:32 +0000"/>
                            <attachment id="180496" name="flushes.png" size="142388" author="bruce.lucas@mongodb.com" created="Wed, 28 Feb 2018 14:49:19 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 28 Feb 2018 14:01:34 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 50 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>bruce.lucas@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 50 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>bruce.lucas@mongodb.com</customfieldvalue>
            <customfieldvalue>tarvip</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htrbif:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htippr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htqxon:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>