<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:48:41 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-38342] Cache hits 95% and performance degrades (like SERVER-26055)</title>
                <link>https://jira.mongodb.org/browse/SERVER-38342</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Recently we have found that the primary is hitting 95% cache usage and query responses become very slow.&lt;/p&gt;

&lt;p&gt;Nothing material is known to have changed (ie data/size/workload etc).&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;I have diagnostics saved from the time of the event and can provide other details as requested.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Following a step down the cache recovered to 80% within a few minutes.&lt;/p&gt;</description>
                <environment></environment>
        <key id="641200">SERVER-38342</key>
            <summary>Cache hits 95% and performance degrades (like SERVER-26055)</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="kelsey.schubert@mongodb.com">Kelsey Schubert</assignee>
                                    <reporter username="paul.ridgway">Paul Ridgway</reporter>
                        <labels>
                    </labels>
                <created>Fri, 30 Nov 2018 19:30:11 +0000</created>
                <updated>Tue, 29 Jan 2019 19:58:38 +0000</updated>
                            <resolved>Mon, 3 Dec 2018 19:54:35 +0000</resolved>
                                    <version>3.4.18</version>
                                                    <component>WiredTiger</component>
                                        <votes>0</votes>
                                    <watches>8</watches>
                                                                                                                <comments>
                            <comment id="2081127" author="thomas.schubert" created="Wed, 5 Dec 2018 14:36:07 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=paul.ridgway&quot; class=&quot;user-hover&quot; rel=&quot;paul.ridgway&quot;&gt;paul.ridgway&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;I&apos;d recommend checking out our docs on profiling, currentOp, and looking at the logs and possibly increasing the verbosity of the logs. Please note that the SERVER project is for reporting bugs or feature suggestions for the MongoDB server. For MongoDB-related support discussion please post on the &lt;a href=&quot;http://groups.google.com/group/mongodb-user&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;mongodb-user group&lt;/a&gt; or &lt;a href=&quot;http://stackoverflow.com/questions/tagged/mongodb&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Stack Overflow with the &lt;tt&gt;mongodb&lt;/tt&gt; tag&lt;/a&gt;. A question like this involving more discussion would be best posted on the mongodb-users group.&lt;/p&gt;

&lt;p&gt;See also our &lt;a href=&quot;https://docs.mongodb.org/manual/support&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Technical Support page&lt;/a&gt; for additional support resources. &lt;/p&gt;

&lt;p&gt;Kind regards,&lt;br/&gt;
Kelsey&lt;/p&gt;</comment>
                            <comment id="2081053" author="paul.ridgway" created="Wed, 5 Dec 2018 13:20:55 +0000"  >&lt;p&gt;Thanks, will take a look. We are struggling to identify the sort of queries that are running during the time - any advice you can give there?&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="2078976" author="thomas.schubert" created="Mon, 3 Dec 2018 19:54:26 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=paul.ridgway&quot; class=&quot;user-hover&quot; rel=&quot;paul.ridgway&quot;&gt;paul.ridgway&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;I&apos;m seeing a very large change in workload, which I suspect is what caused resulting in the cache pressure and subsequent slowdown:&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/202503/202503_workload.png&quot; width=&quot;100%&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;My advice would to investigate the application layer to determine whether this significant change in workload is expected, and if it is whether is can be smoothed.&lt;/p&gt;

&lt;p&gt;Kind regards,&lt;br/&gt;
Kelsey&lt;/p&gt;</comment>
                            <comment id="2077819" author="paul.ridgway" created="Sun, 2 Dec 2018 17:48:58 +0000"  >&lt;p&gt;My bad, I made the same mistake before of removing the @ in the curl command.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;All 3 files should be there now.&lt;/p&gt;</comment>
                            <comment id="2077639" author="thomas.schubert" created="Sat, 1 Dec 2018 15:55:57 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=paul.ridgway&quot; class=&quot;user-hover&quot; rel=&quot;paul.ridgway&quot;&gt;paul.ridgway&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Would you please try uploading these files again? I&apos;m not seeing them in the portal. If you&apos;d like, you could alternatively attach them to this ticket if there is no personal information contained within the logs.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Kelsey&lt;/p&gt;</comment>
                            <comment id="2077565" author="paul.ridgway" created="Sat, 1 Dec 2018 05:28:29 +0000"  >&lt;p&gt;There is a zip from the incident and one from now which may include the recovery. Log is as of now only&lt;/p&gt;</comment>
                            <comment id="2077564" author="paul.ridgway" created="Sat, 1 Dec 2018 05:27:39 +0000"  >&lt;p&gt;Diags and logs uploaded (I think!) - the incident was at 19:24 utc/7:24pm UK time on 30 nov.&lt;/p&gt;</comment>
                            <comment id="2077336" author="thomas.schubert" created="Fri, 30 Nov 2018 21:11:54 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=paul.ridgway&quot; class=&quot;user-hover&quot; rel=&quot;paul.ridgway&quot;&gt;paul.ridgway&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thanks for the report. So we can continue to investigate, would you please provide the logs and diagnostic.data from the affected node? &lt;/p&gt;

&lt;p&gt;I&apos;ve created a &lt;a href=&quot;https://10gen-httpsupload.s3.amazonaws.com/upload_forms/160e0c95-bcdf-49f4-a1bc-4ca0fe489c9e.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;secure upload portal&lt;/a&gt; for you to use. Files uploaded to this portal are only visible to MongoDB employees investigating this issue and are routinely deleted after some time.&lt;/p&gt;

&lt;p&gt;Kind regards,&lt;br/&gt;
Kelsey&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="202503" name="workload.png" size="69674" author="kelsey.schubert@mongodb.com" created="Mon, 3 Dec 2018 19:51:51 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 30 Nov 2018 21:11:54 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 10 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>jmikola@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 10 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>kelsey.schubert@mongodb.com</customfieldvalue>
            <customfieldvalue>paul.ridgway</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hue4sn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hu47x3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                    <customfieldvalue><![CDATA[kelsey.schubert@mongodb.com]]></customfieldvalue>
    

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hudr1z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>