<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:16:08 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-27753] wiredTiger cache 95% performance issues</title>
                <link>https://jira.mongodb.org/browse/SERVER-27753</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Hello, &lt;br/&gt;
We are experiencing performance drop off&apos;s when the wiredTiger Cache becomes 95% full.&lt;/p&gt;

&lt;p&gt;Performing a mongodump on numerous databases serially causes the wiredTiger cache to become 95% populated. &lt;/p&gt;

&lt;p&gt;In some instances the mongodump halts and our backups do not complete. &lt;/p&gt;

&lt;p&gt;What I think is happening is the cache is growing above 95% used. At this point mongo is trying to clear data from the cache and failing, at which point mongodump hangs. &lt;/p&gt;

&lt;p&gt;Changing the size of the wiredTiger cache (up or down) can cause the process to continue, I suspect its causing a forced flush of some cache data. &lt;/p&gt;

&lt;p&gt;Please can you provide a list of data you would like to me to capture and a upload location for this.&lt;/p&gt;

&lt;p&gt;Kind regards, &lt;br/&gt;
Luke&lt;/p&gt;</description>
                <environment>Production</environment>
        <key id="346948">SERVER-27753</key>
            <summary>wiredTiger cache 95% performance issues</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="michael.cahill@mongodb.com">Michael Cahill</assignee>
                                    <reporter username="lukemorfitt">Luke Morfitt</reporter>
                        <labels>
                    </labels>
                <created>Thu, 19 Jan 2017 12:52:43 +0000</created>
                <updated>Fri, 28 Apr 2017 07:21:23 +0000</updated>
                            <resolved>Wed, 1 Mar 2017 05:34:33 +0000</resolved>
                                    <version>3.2.11</version>
                                                    <component>WiredTiger</component>
                                        <votes>0</votes>
                                    <watches>10</watches>
                                                                                                                <comments>
                            <comment id="1559983" author="sumitjha" created="Fri, 28 Apr 2017 07:21:23 +0000"  >&lt;p&gt;Great , Thanks @Thomas&lt;/p&gt;</comment>
                            <comment id="1557175" author="thomas.schubert" created="Tue, 25 Apr 2017 14:18:44 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=sumitjha&quot; class=&quot;user-hover&quot; rel=&quot;sumitjha&quot;&gt;sumitjha&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;The fixes in the patch build that Michael provided earlier are included in MongoDB 3.2.13-rc0, our release candidate for MongoDB 3.2.13, which we intend to release next week. I would recommend using MongoDB 3.2.13-rc0 for your tests as this will provide the best upgrade path to a stable release that contains these fixes.&lt;/p&gt;

&lt;p&gt;Thank you,&lt;br/&gt;
Thomas&lt;/p&gt;</comment>
                            <comment id="1557007" author="sumitjha" created="Tue, 25 Apr 2017 10:37:13 +0000"  >&lt;p&gt;Hi Michael , Can you post the changes you have done over stable release , I will verify that for few another use cases .&lt;/p&gt;</comment>
                            <comment id="1513017" author="michael.cahill" created="Wed, 1 Mar 2017 05:34:33 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lukemorfitt&quot; class=&quot;user-hover&quot; rel=&quot;lukemorfitt&quot;&gt;lukemorfitt&lt;/a&gt;, the changes you need are tracked in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-27700&quot; title=&quot;WT secondary performance drops to near-zero with cache full&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-27700&quot;&gt;&lt;del&gt;SERVER-27700&lt;/del&gt;&lt;/a&gt;, I&apos;m going to close this ticket as a duplicate of that one.&lt;/p&gt;</comment>
                            <comment id="1513012" author="michael.cahill" created="Wed, 1 Mar 2017 04:53:01 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lukemorfitt&quot; class=&quot;user-hover&quot; rel=&quot;lukemorfitt&quot;&gt;lukemorfitt&lt;/a&gt;, thank you for testing the patch build: I&apos;m glad to hear that it improved performance in your workload.&lt;/p&gt;

&lt;p&gt;Here is a summary view from the diagnostic data:&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/150483/150483_server-27753-comparison.png&quot; width=&quot;100%&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;You can see the baseline 3.2.11 stats on the left, and the patched results on the right.  The improvement is stark: with the patched build, there is virtually no time spent waiting for space in cache, and that is because the internal stats show that eviction is orders of magnitude more efficient.&lt;/p&gt;

&lt;p&gt;We are working now to get these changes into a 3.4.x release, once that is stable we will get them into a 3.2.x release.  It will take at least a few weeks for a production-ready release.&lt;/p&gt;</comment>
                            <comment id="1511056" author="lukemorfitt" created="Tue, 28 Feb 2017 09:54:48 +0000"  >&lt;p&gt;Hello,&lt;/p&gt;

&lt;p&gt;Please see attached screenshots. I&apos;ve also uploaded diagnostic data per and post custom binary for your review.&lt;/p&gt;

&lt;p&gt;We placed the custom binary on the 24th. As you can see from the graph&apos;s the performance characteristics have improved significantly.&lt;/p&gt;

&lt;p&gt;WiredTiger cache appears to be evicting more aggressively and not reaching the high water mark and hanging.&lt;/p&gt;

&lt;p&gt;The global locks have dropped from over a second to less than 10ms. &lt;/p&gt;

&lt;p&gt;Are these changes going to be available in a future mongo build?&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;a id=&quot;150411_thumb&quot; href=&quot;https://jira.mongodb.org/secure/attachment/150411/150411_CB+Screen+Shot+2017-02-28+at+09.24.25.png&quot; title=&quot;CB Screen Shot 2017-02-28 at 09.24.25.png&quot; file-preview-type=&quot;image&quot; file-preview-id=&quot;150411&quot; file-preview-title=&quot;CB Screen Shot 2017-02-28 at 09.24.25.png&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/thumbnail/150411/_thumb_150411.png&quot; style=&quot;border: 0px solid black&quot; role=&quot;presentation&quot;/&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;a id=&quot;150412_thumb&quot; href=&quot;https://jira.mongodb.org/secure/attachment/150412/150412_CB+Screen+Shot+2017-02-28+at+09.24.38.png&quot; title=&quot;CB Screen Shot 2017-02-28 at 09.24.38.png&quot; file-preview-type=&quot;image&quot; file-preview-id=&quot;150412&quot; file-preview-title=&quot;CB Screen Shot 2017-02-28 at 09.24.38.png&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/thumbnail/150412/_thumb_150412.png&quot; style=&quot;border: 0px solid black&quot; role=&quot;presentation&quot;/&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;a id=&quot;150413_thumb&quot; href=&quot;https://jira.mongodb.org/secure/attachment/150413/150413_CB+Screen+Shot+2017-02-28+at+09.25.02.png&quot; title=&quot;CB Screen Shot 2017-02-28 at 09.25.02.png&quot; file-preview-type=&quot;image&quot; file-preview-id=&quot;150413&quot; file-preview-title=&quot;CB Screen Shot 2017-02-28 at 09.25.02.png&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/thumbnail/150413/_thumb_150413.png&quot; style=&quot;border: 0px solid black&quot; role=&quot;presentation&quot;/&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;a id=&quot;150414_thumb&quot; href=&quot;https://jira.mongodb.org/secure/attachment/150414/150414_CB+Screen+Shot+2017-02-28+at+09.25.22.png&quot; title=&quot;CB Screen Shot 2017-02-28 at 09.25.22.png&quot; file-preview-type=&quot;image&quot; file-preview-id=&quot;150414&quot; file-preview-title=&quot;CB Screen Shot 2017-02-28 at 09.25.22.png&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/thumbnail/150414/_thumb_150414.png&quot; style=&quot;border: 0px solid black&quot; role=&quot;presentation&quot;/&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;a id=&quot;150415_thumb&quot; href=&quot;https://jira.mongodb.org/secure/attachment/150415/150415_CB+Screen+Shot+2017-02-28+at+09.25.34.png&quot; title=&quot;CB Screen Shot 2017-02-28 at 09.25.34.png&quot; file-preview-type=&quot;image&quot; file-preview-id=&quot;150415&quot; file-preview-title=&quot;CB Screen Shot 2017-02-28 at 09.25.34.png&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/thumbnail/150415/_thumb_150415.png&quot; style=&quot;border: 0px solid black&quot; role=&quot;presentation&quot;/&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;


&lt;p&gt;Kind regards,&lt;br/&gt;
Luke&lt;/p&gt;</comment>
                            <comment id="1506914" author="lukemorfitt" created="Thu, 23 Feb 2017 09:58:19 +0000"  >&lt;p&gt;Michael Cahill, thank you for the custom build. &lt;/p&gt;

&lt;p&gt;We will perform some testing and back back to you.&lt;/p&gt;

&lt;p&gt;Kind regards,&lt;br/&gt;
Luke&lt;/p&gt;</comment>
                            <comment id="1504671" author="michael.cahill" created="Tue, 21 Feb 2017 05:40:49 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lukemorfitt&quot; class=&quot;user-hover&quot; rel=&quot;lukemorfitt&quot;&gt;lukemorfitt&lt;/a&gt;, thank you for uploading the diagnostic data, I have looked through it.&lt;/p&gt;

&lt;p&gt;Your case looks similar to what we have seen in several workloads recently.  In particular, there are over 20,000 active collections + indexes, and in that case with some workloads we have seen WiredTiger&apos;s cache management become inefficient.  See &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-27700&quot; title=&quot;WT secondary performance drops to near-zero with cache full&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-27700&quot;&gt;&lt;del&gt;SERVER-27700&lt;/del&gt;&lt;/a&gt; for more details, the short version is that eviction is wasting time looking for pages to evict in trees that occupy very little space in cache.&lt;/p&gt;

&lt;p&gt;We have been working to improve WiredTiger for this type of workload (see &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-27700?focusedCommentId=1484627&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-1484627&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;here&lt;/a&gt; for a summary of the changes).&lt;/p&gt;

&lt;p&gt;Are you able to test a patch build of MongoDB with your workload and measure whether it improves performance in this situation for you?  If so, a build for Ubuntu 14.04 is &lt;a href=&quot;https://s3.amazonaws.com/mciuploads/mongodb-mongo-v3.2/ubuntu1404/95d4a0a472e436dad9380db917145f121d8c3720/binaries/mongo-mongodb_mongo_v3.2_ubuntu1404_95d4a0a472e436dad9380db917145f121d8c3720_17_02_20_04_43_56.tgz&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="1485717" author="lukemorfitt" created="Thu, 26 Jan 2017 10:20:29 +0000"  >&lt;p&gt;Cache usage graphs&lt;/p&gt;</comment>
                            <comment id="1485715" author="lukemorfitt" created="Thu, 26 Jan 2017 10:19:58 +0000"  >&lt;p&gt;Hello ~mark.agarunov,&lt;/p&gt;


&lt;p&gt;It feels like the cache eviction threads are not able to keep up with the incoming data and thus the performance drops off.  Is it possible to tune the aggressiveness of the cache eviction process?&lt;/p&gt;

&lt;p&gt;As the cache reaches 95% usage the performance of the server drops off and the lock times increase. &lt;br/&gt;
&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;a id=&quot;148300_thumb&quot; href=&quot;https://jira.mongodb.org/secure/attachment/148300/148300_Cache+Filling+and+hitting+95+image+1.png&quot; title=&quot;Cache Filling and hitting 95 image 1.png&quot; file-preview-type=&quot;image&quot; file-preview-id=&quot;148300&quot; file-preview-title=&quot;Cache Filling and hitting 95 image 1.png&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/thumbnail/148300/_thumb_148300.png&quot; style=&quot;border: 0px solid black&quot; role=&quot;presentation&quot;/&gt;&lt;/a&gt;&lt;/span&gt;&lt;br/&gt;
&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;a id=&quot;148301_thumb&quot; href=&quot;https://jira.mongodb.org/secure/attachment/148301/148301_Cache+filling+and+hitting+95+images+2.png&quot; title=&quot;Cache filling and hitting 95 images 2.png&quot; file-preview-type=&quot;image&quot; file-preview-id=&quot;148301&quot; file-preview-title=&quot;Cache filling and hitting 95 images 2.png&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/thumbnail/148301/_thumb_148301.png&quot; style=&quot;border: 0px solid black&quot; role=&quot;presentation&quot;/&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;Resizing the cache allows mongo to continue until it reaches the new 95% level.&lt;br/&gt;
&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;a id=&quot;148302_thumb&quot; href=&quot;https://jira.mongodb.org/secure/attachment/148302/148302_Resized+cache+image1.png&quot; title=&quot;Resized cache image1.png&quot; file-preview-type=&quot;image&quot; file-preview-id=&quot;148302&quot; file-preview-title=&quot;Resized cache image1.png&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/thumbnail/148302/_thumb_148302.png&quot; style=&quot;border: 0px solid black&quot; role=&quot;presentation&quot;/&gt;&lt;/a&gt;&lt;/span&gt;&lt;br/&gt;
&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;a id=&quot;148303_thumb&quot; href=&quot;https://jira.mongodb.org/secure/attachment/148303/148303_Resized+cahce+image+2.png&quot; title=&quot;Resized cahce image 2.png&quot; file-preview-type=&quot;image&quot; file-preview-id=&quot;148303&quot; file-preview-title=&quot;Resized cahce image 2.png&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/thumbnail/148303/_thumb_148303.png&quot; style=&quot;border: 0px solid black&quot; role=&quot;presentation&quot;/&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;



&lt;p&gt;The server discussed within this post is a secondary in a 3 node replica set (r3.8xlarge, 32cores, 244gb ram, 1Tb data disk, 334gb Journel disk).  It is storing 326gb on the datadisk with a 50gb OpLog. It is hosting 176 databases, only 14 databases consume and significant space ( more than 1gb in size) as per below.&lt;/p&gt;

&lt;p&gt;dbname 1.173GB&lt;br/&gt;
dbname 1.374GB&lt;br/&gt;
dbname 1.407GB&lt;br/&gt;
dbname 1.839GB&lt;br/&gt;
dbname 2.060GB&lt;br/&gt;
dbname 23.149GB&lt;br/&gt;
local 26.939GB&lt;br/&gt;
dbname 4.240GB&lt;br/&gt;
dbname 48.836GB&lt;br/&gt;
dbname 6.272GB&lt;br/&gt;
dbname 60.971GB&lt;br/&gt;
dbname 66.852GB&lt;br/&gt;
dbname 67.230GB&lt;br/&gt;
dbname 7.656GB&lt;/p&gt;

&lt;p&gt;I&apos;ve uploaded the following files to the portal.&lt;/p&gt;

&lt;p&gt;diagnostic.data.tar.gz&lt;br/&gt;
mongod.log.cache_hits_95.gz&lt;br/&gt;
mongod.log.cache_resize.gz&lt;/p&gt;


&lt;p&gt;Kind regards,&lt;br/&gt;
Luke&lt;/p&gt;</comment>
                            <comment id="1481333" author="mark.agarunov" created="Fri, 20 Jan 2017 16:16:10 +0000"  >&lt;p&gt;Hello &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lukemorfitt&quot; class=&quot;user-hover&quot; rel=&quot;lukemorfitt&quot;&gt;lukemorfitt&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;I&apos;ve created a &lt;a href=&quot;https://10gen-httpsupload.s3.amazonaws.com/upload_forms/2a9b9cc1-a392-4264-9a0c-31bde1e86ad6.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;safe, secure upload portal&lt;/a&gt; so your logs can only be accessed by us for the purpose of debugging this issue. Just to reiterate:&lt;/p&gt;

&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;Please archive (tar or zip) the &lt;tt&gt;$dbpath/diagnostic.data&lt;/tt&gt; directory and upload the archive via the portal.&lt;/li&gt;
	&lt;li&gt;Please upload the logs from mongod for the time period when this issue was present.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Thanks,&lt;br/&gt;
Mark&lt;/p&gt;</comment>
                            <comment id="1481100" author="lukemorfitt" created="Fri, 20 Jan 2017 09:46:47 +0000"  >&lt;p&gt;Hello Mark,&lt;/p&gt;

&lt;p&gt;Thank you for the update. I will gather the information for you.&lt;/p&gt;

&lt;p&gt;Please can you provide a secure upload location as our server logs will contain sensitive information.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Luke&lt;/p&gt;</comment>
                            <comment id="1480737" author="mark.agarunov" created="Thu, 19 Jan 2017 21:53:25 +0000"  >&lt;p&gt;Hello &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lukemorfitt&quot; class=&quot;user-hover&quot; rel=&quot;lukemorfitt&quot;&gt;lukemorfitt&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thank you for the report. This may be expected behavior; at 95% full, some of the application threads are used to do cache evictions which causes a slowdown. However, to definitively determine if this is due to a bug or expected behavior, please provide the following if possible:&lt;/p&gt;

&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;Please archive (tar or zip) the &lt;tt&gt;$dbpath/diagnostic.data&lt;/tt&gt; directory and attach it to this ticket&lt;/li&gt;
	&lt;li&gt;Please attach the logs from mongod for the time period when this issue was present.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Thanks,&lt;br/&gt;
Mark&lt;/p&gt;</comment>
                            <comment id="1480184" author="lukemorfitt" created="Thu, 19 Jan 2017 13:48:20 +0000"  >&lt;p&gt;We are also seeing this issue whilst trying to add a new hidden node to the replicaset. &lt;/p&gt;

&lt;p&gt;The new node is trying to replicate data from a mongo secondary server. Once this server hits 95% cache utilization (105gb cache) the replicating node slows down whilst replicating data from the secondary.&lt;/p&gt;

&lt;p&gt;Down sizing the cache from 105G to 104G allowed the replication to continue. &lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;insert query update delete getmore command % dirty % used flushes  vsize   res qr|qw ar|aw  netIn netOut conn             set repl                 time&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  *180    *0   *769    *35       0     4|0     0.3   95.0       0   105G 96.8G   0|0  0|16   503b  23.1k   37 repl-gsp-aeu002  SEC 2017-01-19T13:48:52Z&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  *290    41  *1113    *68       1     2|0     0.3   95.0       0   105G 96.8G   0|0   0|1  7.57k  30.7k   37 repl-gsp-aeu002  SEC 2017-01-19T13:48:53Z&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  *228    *0   *843   *103       0     3|0     0.3   95.0       0   105G 96.8G   0|0  0|16   445b  22.7k   37 repl-gsp-aeu002  SEC 2017-01-19T13:48:54Z&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Cache was downscaled here.&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;*1512   100  *6017  *3507       0     3|0     0.4   91.5       0   105G 96.8G   0|0  0|16  17.8k  47.3k   37 repl-gsp-aeu002  SEC 2017-01-19T13:48:55Z&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;*7032   402 *27015  *3692       0     4|0     0.7   91.7       0   105G 96.8G   0|0   1|0  71.6k  98.3k   37 repl-gsp-aeu002  SEC 2017-01-19T13:48:56Z&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  *74     2   *438    *10      14     8|0     0.7   91.8       0   105G 96.8G   0|0   1|0  1.10k  45.4k   35 repl-gsp-aeu002  SEC 2017-01-19T13:48:57Z&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; *106    *0   *454     *9      11     3|0     0.7   91.8       0   105G 96.8G   0|1   0|0   445b  22.7k   35 repl-gsp-aeu002  SEC 2017-01-19T13:48:58Z&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;thanks,&lt;br/&gt;
Luke&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="346241">SERVER-27700</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="333604">SERVER-27153</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="364898">SERVER-28316</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="150411" name="CB Screen Shot 2017-02-28 at 09.24.25.png" size="85307" author="lukemorfitt" created="Tue, 28 Feb 2017 09:28:06 +0000"/>
                            <attachment id="150412" name="CB Screen Shot 2017-02-28 at 09.24.38.png" size="81977" author="lukemorfitt" created="Tue, 28 Feb 2017 09:28:06 +0000"/>
                            <attachment id="150413" name="CB Screen Shot 2017-02-28 at 09.25.02.png" size="143097" author="lukemorfitt" created="Tue, 28 Feb 2017 09:28:06 +0000"/>
                            <attachment id="150414" name="CB Screen Shot 2017-02-28 at 09.25.22.png" size="112425" author="lukemorfitt" created="Tue, 28 Feb 2017 09:28:06 +0000"/>
                            <attachment id="150415" name="CB Screen Shot 2017-02-28 at 09.25.34.png" size="102081" author="lukemorfitt" created="Tue, 28 Feb 2017 09:28:06 +0000"/>
                            <attachment id="148300" name="Cache Filling and hitting 95 image 1.png" size="106613" author="lukemorfitt" created="Thu, 26 Jan 2017 10:20:23 +0000"/>
                            <attachment id="148301" name="Cache filling and hitting 95 images 2.png" size="58317" author="lukemorfitt" created="Thu, 26 Jan 2017 10:20:22 +0000"/>
                            <attachment id="148302" name="Resized cache image1.png" size="111674" author="lukemorfitt" created="Thu, 26 Jan 2017 10:20:22 +0000"/>
                            <attachment id="148303" name="Resized cahce image 2.png" size="59870" author="lukemorfitt" created="Thu, 26 Jan 2017 10:20:22 +0000"/>
                            <attachment id="150483" name="server-27753-comparison.png" size="161014" author="michael.cahill@mongodb.com" created="Wed, 1 Mar 2017 04:52:56 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>14.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 19 Jan 2017 21:53:25 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 41 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>sumitjha</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 41 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>kelsey.schubert@mongodb.com</customfieldvalue>
            <customfieldvalue>lukemorfitt</customfieldvalue>
            <customfieldvalue>mark.agarunov</customfieldvalue>
            <customfieldvalue>michael.cahill@mongodb.com</customfieldvalue>
            <customfieldvalue>sumitjha</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht177b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hstwpb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                    <customfieldvalue><![CDATA[kelsey.schubert@mongodb.com]]></customfieldvalue>
    

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hs4ijb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>