<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:06:45 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-4710] Writers queues never go down and server eventually stalls</title>
                <link>https://jira.mongodb.org/browse/SERVER-4710</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We&apos;ve posted on mongo-users what we believe is our core issue right now, namely the queued writers which never seem to go down once they&apos;ve reached a threshold. &lt;/p&gt;

&lt;p&gt;The thread is here: &lt;a href=&quot;http://groups.google.com/group/mongodb-user/browse_thread/thread/195264a598d87393#&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://groups.google.com/group/mongodb-user/browse_thread/thread/195264a598d87393#&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;We&apos;ve been able to reproduce the problem yesterday. After a --repair, the server seemed fine for a while, and when copying on the server the log file (the -v switch proved itself very verbose indeed, we had over 2GB of logs for a few hours of activity), the qw figure shot up drastically, as did the inbound connections (the clients were trying to compensate for the blocking queries), right before the server stopped serving queries altogether. &lt;/p&gt;

&lt;p&gt;I&apos;ve uploaded on jira part of the logs for this episode...&lt;/p&gt;

&lt;p&gt;Later on, when we issued a repair on the database, the log was&lt;/p&gt;

&lt;p&gt;Tue Jan 17 00:09:45 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; warning: ClientCursor::yield can&apos;t unlock b/c of recursive lock ns: Main.cachedItems top: { opid: 220, a&lt;br/&gt;
ctive: true, waitingForLock: false, secs_running: 0, op: &quot;getmore&quot;, ns: &quot;Main.cachedItems&quot;, query: {}, client: &quot;0.0.0.0:0&quot;, desc: &quot;initandli&lt;br/&gt;
sten&quot;, numYields: 0 }&lt;br/&gt;
Tue Jan 17 00:09:46 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; warning: ClientCursor::yield can&apos;t unlock b/c of recursive lock ns: Main.cachedItems top: { opid: 221, a&lt;br/&gt;
ctive: true, waitingForLock: false, secs_running: 0, op: &quot;getmore&quot;, ns: &quot;Main.cachedItems&quot;, query: {}, client: &quot;0.0.0.0:0&quot;, desc: &quot;initandli&lt;br/&gt;
sten&quot;, numYields: 0 }&lt;/p&gt;


&lt;p&gt;(and so on for the 223 stuck queued writers).&lt;/p&gt;

&lt;p&gt;So it looks like some of our queries (we&apos;re implementing a poor-man distributed lock using atomic sets and checks) may be problematic when mongod is under duress (IO starvation, or other factors). We&apos;re moving the lock mechanism to a dedicated instance, and we&apos;re working more generally on our data access profile to minimize writes wrt reads, but we need to find out the root causes for this incident.&lt;/p&gt;
</description>
                <environment>Windows x64</environment>
        <key id="28591">SERVER-4710</key>
            <summary>Writers queues never go down and server eventually stalls</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="4">Incomplete</resolution>
                                        <assignee username="milkie@mongodb.com">Eric Milkie</assignee>
                                    <reporter username="kbjmongo">YannPierre CouzySchwartz</reporter>
                        <labels>
                    </labels>
                <created>Wed, 18 Jan 2012 15:28:10 +0000</created>
                <updated>Wed, 15 Aug 2012 14:24:15 +0000</updated>
                            <resolved>Fri, 2 Mar 2012 19:32:15 +0000</resolved>
                                    <version>2.0.2</version>
                                                    <component>Stability</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="94786" author="prod@kobojo.com" created="Fri, 2 Mar 2012 21:54:16 +0000"  >&lt;p&gt;Sorry for not giving updates. We&apos;ve migrated to 2.1.0 (2008+) and the locking problem has never reappeared. We later switched to 2.0.3 and our issue is fixed for this version as well. Thank you.&lt;/p&gt;</comment>
                            <comment id="93487" author="milkie" created="Tue, 28 Feb 2012 19:46:30 +0000"  >&lt;p&gt;How&apos;s your system running now with the Windows2008R2 build?&lt;/p&gt;</comment>
                            <comment id="85740" author="milkie" created="Mon, 6 Feb 2012 18:47:45 +0000"  >&lt;p&gt;I tried using a build of 2.0.3 that is very close to the one you are using and still can&apos;t reproduce.  Since I believe the code is essentially the same, the cause of the issue may be either the hardware, or the format or content of your data.&lt;/p&gt;

&lt;p&gt;What parameters are you using to start mongod?  I did try raising the verbosity level on mine to see if the extra info would help, but unfortunately it logs every insert during the import, and thus slowed it down significantly.&lt;br/&gt;
Do you see anything abnormal in the mongod log, compared to what is shown for the import using 2.0?&lt;/p&gt;

&lt;p&gt;Other than this mongorestore issue, is it correct to say that it is running fine with the new Windows2008+ build?&lt;/p&gt;</comment>
                            <comment id="85685" author="kbjmongo" created="Mon, 6 Feb 2012 16:44:16 +0000"  >&lt;p&gt;Mongorestore with --host --db and -c switches (I&apos;ve tried it with and without --drop)&lt;br/&gt;
The file to restore is on the same drive.&lt;/p&gt;

&lt;p&gt;The binary we tested was compiled on Jan 30th &lt;/p&gt;

&lt;p&gt;Running mongod we get:&lt;br/&gt;
Mon Feb 06 16:42:45 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; db version v2.0.3-pre-, pdfile version 4.5&lt;br/&gt;
Mon Feb 06 16:42:45 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; git version: ab1e828dd777945a0afe67c66c37ca2deb2c6ee1&lt;br/&gt;
Mon Feb 06 16:42:45 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; build info: windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack=&apos;Service Pack 1&apos;) BOOST_LIB_VERSION=1_42&lt;/p&gt;</comment>
                            <comment id="85667" author="milkie" created="Mon, 6 Feb 2012 16:31:28 +0000"  >&lt;p&gt;I tried dumping a 300MB database with 2.0.2 and then restored it with mongorestore using a Nightly build of 2.1.1/Windows 2008.  The restore completes in 1 min 15 secs.  There may be a regression in the specific build you are using.  Can you give the initial log messages when you start mongod?  Within the messages is the &quot;git version&quot; and &quot;db version&quot; which will point me to the specific build you are using.&lt;br/&gt;
Are you using any command line parameters to mongorestore, other than the path to the dump? &lt;/p&gt;</comment>
                            <comment id="85560" author="kbjmongo" created="Mon, 6 Feb 2012 10:39:36 +0000"  >&lt;p&gt;Restoring on a 2.0.2 then running the latest binary pointing to the data directory works fine though. The issue seems to be with the way mongorestore works in the latest build you provided.&lt;/p&gt;</comment>
                            <comment id="85559" author="kbjmongo" created="Mon, 6 Feb 2012 10:30:52 +0000"  >&lt;p&gt;We did a few tests with the 2008R2 build, but when we tried to use mongorestore to restore our data, the restore process was very very slow (less than 10MB a minute, after 15 minutes it was still showing 0% progress)... We tested this on a 2GB and a 6GB dumps. &lt;/p&gt;

&lt;p&gt;The restore on the same backup files is very fast with the current 2.0.2.&lt;/p&gt;</comment>
                            <comment id="81719" author="eliot" created="Mon, 23 Jan 2012 18:49:41 +0000"  >&lt;p&gt;Which build was that with in production?&lt;/p&gt;

&lt;p&gt;CAn you switch to the 2008R2+ build?&lt;/p&gt;</comment>
                            <comment id="81696" author="kbjmongo" created="Mon, 23 Jan 2012 18:10:40 +0000"  >&lt;p&gt;We haven&apos;t been able to reproduce the problem on our test setups. But we have in production. The relevant logs have been uploaded in the community private jira.&lt;/p&gt;</comment>
                            <comment id="80750" author="kbjmongo" created="Wed, 18 Jan 2012 17:35:21 +0000"  >&lt;p&gt;Mmh. The nightly build requires 2008 R2 (TryAcquireSRWLockExclusive), I&apos;ll need to switch the OS before testing, more info when everything is set up (a few hours from now).&lt;/p&gt;</comment>
                            <comment id="80725" author="kbjmongo" created="Wed, 18 Jan 2012 15:42:21 +0000"  >&lt;p&gt;Thank you Eliot. We&apos;ll try to reproduce on our current dev setup (2.0.2) then we&apos;ll try out your build.&lt;/p&gt;</comment>
                            <comment id="80721" author="eliot" created="Wed, 18 Jan 2012 15:32:54 +0000"  >&lt;p&gt;There is a newer windows build that might help.&lt;br/&gt;
Can you try this:&lt;br/&gt;
&lt;a href=&quot;http://downloads.mongodb.org/win32/mongodb-win32-x86_64-2008plus-v2.0-latest.zip&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://downloads.mongodb.org/win32/mongodb-win32-x86_64-2008plus-v2.0-latest.zip&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>12.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 18 Jan 2012 15:32:54 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        11 years, 50 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            11 years, 50 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10022"><![CDATA[Windows]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>eliot</customfieldvalue>
            <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>kbjmongo</customfieldvalue>
            <customfieldvalue>prod@kobojo.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrogk7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hritof:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>23461</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrnuuv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>