<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:01:57 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-22964] IX GlobalLock being held while wating for wt cache eviction</title>
                <link>https://jira.mongodb.org/browse/SERVER-22964</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;hi,&lt;br/&gt;
I&apos;m currently doing a pressure test for 3.2.3 using the attached test script. The testing environment is a two nodes replSet using WiredTiger. After running the script for long time(about 2 days). The mongod process will hang. As you can see in the attached thread stack, a lot of threads were waiting for the GlobalLock which was being held by someone(probably the thread doing profile, i.e. Thread 501). Looks like the thread locked it with MODE_IX and was doing a cappedDelete operation, making room for new profiling document. But it seems that it was waiting on WT&apos;s cache evict_waiter_cond and hangs there forever.&lt;/p&gt;</description>
                <environment></environment>
        <key id="269948">SERVER-22964</key>
            <summary>IX GlobalLock being held while wating for wt cache eviction</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="michael.cahill@mongodb.com">Michael Cahill</assignee>
                                    <reporter username="zhcn381">CenZheng</reporter>
                        <labels>
                            <label>WTplaybook</label>
                            <label>code-and-test</label>
                    </labels>
                <created>Fri, 4 Mar 2016 12:43:41 +0000</created>
                <updated>Sat, 19 Nov 2016 20:29:31 +0000</updated>
                            <resolved>Wed, 30 Mar 2016 01:27:28 +0000</resolved>
                                    <version>3.2.3</version>
                                    <fixVersion>3.0.12</fixVersion>
                    <fixVersion>3.2.5</fixVersion>
                    <fixVersion>3.3.4</fixVersion>
                                    <component>WiredTiger</component>
                                        <votes>0</votes>
                                    <watches>17</watches>
                                                                                                                <comments>
                            <comment id="1199912" author="zhcn381" created="Fri, 11 Mar 2016 01:35:53 +0000"  >&lt;p&gt;Get it, Thank you Ramon~&lt;/p&gt;</comment>
                            <comment id="1198906" author="ramon.fernandez" created="Thu, 10 Mar 2016 12:36:37 +0000"  >&lt;p&gt;Thanks for the additional information &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=zhcn381&quot; class=&quot;user-hover&quot; rel=&quot;zhcn381&quot;&gt;zhcn381&lt;/a&gt;, we believe we&apos;ve identified the source of the problem and have produced a fix in &lt;a href=&quot;https://jira.mongodb.org/browse/WT-2451&quot; title=&quot;Allow eviction of metadata&quot; class=&quot;issue-link&quot; data-issue-key=&quot;WT-2451&quot;&gt;&lt;del&gt;WT-2451&lt;/del&gt;&lt;/a&gt;. Here&apos;s the explanation from a colleague:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;What&apos;s causing the issue is the large amount of create and drop operations: each time a create or drop is done we update the WiredTiger metadata. The metadata is currently never evicted from the cache which means that we never remove entries created for deleted items, so if there are enough create/drop operations the metadata can grow quite large. If the cache is reasonably constrained we can end up with a situation where the metadata fills the entire cache and prevents other operations from completing.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;This large amount of create and drop operations is not very common in production cases, but not unheard of either. If this will be your production use case, until this fix makes it into a stable release note that using a larger WiredTiger cache will easily help you work around the issue.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                            <comment id="1198600" author="zhcn381" created="Thu, 10 Mar 2016 02:22:36 +0000"  >&lt;p&gt;Hi, Ramon&lt;/p&gt;

&lt;p&gt;The situation occurred again, I have uploaded the latest diagnostics data(metrics.2016-03-09T13-01-56Z-00000.tar.gz). You can have a look.&lt;/p&gt;

&lt;p&gt;Thanks~&lt;/p&gt;</comment>
                            <comment id="1196194" author="ramon.fernandez" created="Tue, 8 Mar 2016 12:51:12 +0000"  >&lt;p&gt;Thanks for uploading &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=zhcn381&quot; class=&quot;user-hover&quot; rel=&quot;zhcn381&quot;&gt;zhcn381&lt;/a&gt;, I can see the diagnostics data and I&apos;m taking a look.&lt;/p&gt;</comment>
                            <comment id="1196002" author="zhcn381" created="Tue, 8 Mar 2016 06:24:34 +0000"  >&lt;p&gt;hi Ramon, I have uploaded last time&apos;s diagnostic.data file, pls check! &lt;/p&gt;

&lt;p&gt;Thanks&lt;/p&gt;</comment>
                            <comment id="1194796" author="ramon.fernandez" created="Mon, 7 Mar 2016 12:36:08 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=zhcn381&quot; class=&quot;user-hover&quot; rel=&quot;zhcn381&quot;&gt;zhcn381&lt;/a&gt;, I have been running your repro script for almost 48h but my setup hasn&apos;t hung. We do have some ideas about what the issue could be, but if it is possible for you to upload the &lt;tt&gt;diagnostic.data&lt;/tt&gt; folder this information would still be useful investigating this ticket.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                            <comment id="1194002" author="zhcn381" created="Sat, 5 Mar 2016 14:54:43 +0000"  >&lt;p&gt;hi Ramon,&lt;br/&gt;
I can&apos;t upload the diagnositc.data directory due to the poor network status. I&apos;ll try on next Monday if you still need it then. &lt;br/&gt;
Thanks&lt;/p&gt;</comment>
                            <comment id="1193020" author="ramon.fernandez" created="Fri, 4 Mar 2016 13:37:39 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=zhcn381&quot; class=&quot;user-hover&quot; rel=&quot;zhcn381&quot;&gt;zhcn381&lt;/a&gt;, can you please &lt;a href=&quot;https://10gen-httpsupload.s3.amazonaws.com/upload_forms/cc047ddb-bee8-4446-a5c5-0c5d5178e132.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;upload here&lt;/a&gt; the contents of the &lt;tt&gt;diagnostic.data&lt;/tt&gt; directory from your &lt;tt&gt;dbpath&lt;/tt&gt; and the full log file of the affected node?&lt;/p&gt;

&lt;p&gt;I&apos;ll try to reproduce this behavior on my end in the mean time, but since you say it takes two days it will be faster to look at the diagnostic data already captured by your system.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="270330">WT-2451</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="112086" name="diags1.tgz" size="17996567" author="ramon.fernandez@mongodb.com" created="Sat, 5 Mar 2016 14:27:15 +0000"/>
                            <attachment id="111886" name="global_lock_hang.ts" size="1271586" author="zhcn381" created="Fri, 4 Mar 2016 12:43:41 +0000"/>
                            <attachment id="111885" name="mongod.conf" size="355" author="zhcn381" created="Fri, 4 Mar 2016 12:43:41 +0000"/>
                            <attachment id="111887" name="test.sh" size="821" author="zhcn381" created="Fri, 4 Mar 2016 12:43:41 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12451" key="com.atlassian.jira.plugin.system.customfieldtypes:multiversion">
                        <customfieldname>Backport Completed</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="16680">3.0.12</customfieldvalue>
    <customfieldvalue id="16541">3.2.5</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 4 Mar 2016 13:37:39 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        7 years, 48 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/WT-2451'>WT-2451</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            7 years, 48 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10020"><![CDATA[Linux]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>zhcn381</customfieldvalue>
            <customfieldvalue>michael.cahill@mongodb.com</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrkejb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hsj21r:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;1. setup a two node replSet using the attached mongod.conf(only specify storage related)&lt;br/&gt;
2.  run test.sh&lt;br/&gt;
3. It will hang after running the script for about a long time(2 days+)&lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hschjz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>