<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:13:15 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-6965] mongo process going unresponsive</title>
                <link>https://jira.mongodb.org/browse/SERVER-6965</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Replica set with 5 mongo boxes (slaveOk = disabled)&lt;/p&gt;

&lt;p&gt;Suddenly the mongod process go unresponsive. The box just keeps on getting connections from other mongod servers or applications(causing the num-sockets to keep on increasing). Also at that time, I am not able to login to mongo console. Neither it gets stopped from our regular stop script. The mongod process simply go unresponsive. &lt;/p&gt;

&lt;p&gt;This had happened some 5-6 times with different servers(sometimes on primaries and secondary) last 1 week. And at that time I just have to force kill that process and then restart.&lt;/p&gt;

&lt;p&gt;This is the primary log, where we can clearly see, that suddenly the read queries stopped coming(logging).  &lt;br/&gt;
&lt;a href=&quot;http://pastebin.com/VLciscv7&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://pastebin.com/VLciscv7&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;At the same time, mongo-java-client reporting timeOut exception : &lt;a href=&quot;http://pastebin.com/CB7hEced&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://pastebin.com/CB7hEced&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Also this is not the fact that queries took more time. The queries hardly take 100 ms. Neither there is any spikes in CPU, load (except for the total-open-sockets)&lt;/p&gt;

&lt;p&gt;Need some answer for this behavior?&lt;br/&gt;
Let me know if some more specific information needed.&lt;/p&gt;</description>
                <environment>AWS xLarge instance backed by RAID 10 EBS volumes</environment>
        <key id="49701">SERVER-6965</key>
            <summary>mongo process going unresponsive</summary>
                <type id="6" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14720&amp;avatarType=issuetype">Question</type>
                                            <priority id="1" iconUrl="https://jira.mongodb.org/images/icons/priorities/blocker.svg">Blocker - P1</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="4">Incomplete</resolution>
                                        <assignee username="randolph@mongodb.com">Randolph Tan</assignee>
                                    <reporter username="abhishekiitg10@gmail.com">Abhishek Kumar</reporter>
                        <labels>
                    </labels>
                <created>Fri, 7 Sep 2012 12:27:14 +0000</created>
                <updated>Fri, 8 Mar 2013 15:55:43 +0000</updated>
                            <resolved>Wed, 19 Dec 2012 15:33:49 +0000</resolved>
                                    <version>2.0.2</version>
                                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="174673" author="renctan" created="Mon, 15 Oct 2012 00:16:16 +0000"  >&lt;p&gt;Sorry for the delay, it&apos;s really hard to tell what is going on without the other data either from MMS or mongostats during the hanged state (and a couple of moments before it). It&apos;s also hard to tell without currentOp which operation is the culprit for hanging the server. I would caution against using eval functionalities (and other features that rely on the javascript engine) because it can easily create bottlenecks. This is because it is possible for multiple readers in different threads to hold the read lock while it is not the case for the global Javascript interpreter lock in the server.&lt;/p&gt;</comment>
                            <comment id="174174" author="abhishekiitg10@gmail.com" created="Fri, 12 Oct 2012 11:22:06 +0000"  >&lt;p&gt;Hey any updates on this issue. &lt;br/&gt;
Though I have removed using stored function in system.js function. But I just want to know the problems that might have occurred causing such issue.&lt;/p&gt;</comment>
                            <comment id="163521" author="abhishekiitg10@gmail.com" created="Wed, 12 Sep 2012 16:36:29 +0000"  >&lt;p&gt;Its a stored function in system.js collection, which is called from the application, with nolock = true. &lt;br/&gt;
Yes, the if-else don&apos;t involve any access to database.&lt;/p&gt;

&lt;p&gt;Also, can you update the doc on the page &lt;a href=&quot;http://www.mongodb.org/display/DOCS/Server-side+Code+Execution&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://www.mongodb.org/display/DOCS/Server-side+Code+Execution&lt;/a&gt; regarding nolock option. Its not clear what does this option mean. Will this option ensures that no write lock is taken?&lt;/p&gt;</comment>
                            <comment id="163480" author="renctan" created="Wed, 12 Sep 2012 15:32:37 +0000"  >&lt;p&gt;Sorry, I forgot to ask for some clarifications regarding on how you use the stored function. I&apos;m also assuming that &quot;// some if-else operation&quot; section does not access the database, correct?&lt;/p&gt;</comment>
                            <comment id="163468" author="abhishekiitg10@gmail.com" created="Wed, 12 Sep 2012 15:13:49 +0000"  >&lt;p&gt;Then in the $eval I am just using the read only which dont involve, $where, $eval, map reduce or writes. I still didn&apos;t get the reason for the deadlock. &lt;br/&gt;
Please elaborate for the possibility of the deadlock in the example query that I pasted.&lt;/p&gt;</comment>
                            <comment id="163461" author="renctan" created="Wed, 12 Sep 2012 15:07:35 +0000"  >&lt;p&gt;You can do reads as long as they don&apos;t use the server side js features like $where, $eval, map reduce, but not writes. Thanks for bringing this to our attention, we&apos;ll update our docs to include some warning against using $eval.&lt;/p&gt;</comment>
                            <comment id="162798" author="abhishekiitg10@gmail.com" created="Tue, 11 Sep 2012 02:30:28 +0000"  >&lt;p&gt;So, inside the eval we should never do any read or write operation?&lt;/p&gt;

&lt;p&gt;If this is the case, the docs at &lt;a href=&quot;http://www.mongodb.org/display/DOCS/Server-side+Code+Execution&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://www.mongodb.org/display/DOCS/Server-side+Code+Execution&lt;/a&gt; is not specifying this thing clearly.&lt;/p&gt;</comment>
                            <comment id="162363" author="renctan" created="Mon, 10 Sep 2012 14:20:12 +0000"  >&lt;p&gt;The sample code you posted looks fine since the function is inactive and will not be executed in the server implicitly. The problem with the js features like $eval, $where or mapreduce is you shouldn&apos;t have a function body that either needs a write lock or calls one of the js features. This is because these js features requires the global javascript lock and you can cause a deadlock by doing so.&lt;/p&gt;</comment>
                            <comment id="161887" author="abhishekiitg10@gmail.com" created="Sat, 8 Sep 2012 05:43:48 +0000"  >&lt;p&gt;Its a system.js function which is internally doing two find operations on two different collection. Each find operations on some selected set of documents. &lt;br/&gt;
Can you explain me what kind of deadlock may occur because I am not doing any write operation inside that function.&lt;/p&gt;

&lt;p&gt;$eval function : &lt;a href=&quot;http://pastebin.com/9wHFMQZX&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://pastebin.com/9wHFMQZX&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Also this sometimes happened in secondary which didn&apos;t receive any read operation.&lt;/p&gt;</comment>
                            <comment id="161886" author="renctan" created="Sat, 8 Sep 2012 05:25:41 +0000"  >&lt;p&gt;Depending on how you wrote $eval, you can deadlock the server.&lt;/p&gt;</comment>
                            <comment id="161880" author="abhishekiitg10@gmail.com" created="Sat, 8 Sep 2012 02:17:27 +0000"  >&lt;p&gt;As I told I was not able to connect to mongod via mongo console, so I can&apos;t give you the db.currentOp(). The mongod process simply goes unresponsive.&lt;/p&gt;

&lt;p&gt;The servers are not at MMS.&lt;/p&gt;

&lt;p&gt;I see the following warning all over my mongod logs. &lt;br/&gt;
Wed Sep  5 07:04:21 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn847592&amp;#93;&lt;/span&gt; warning: ClientCursor::yield can&apos;t unlock b/c of recursive lock ns: history.messages top: &lt;/p&gt;
{ err: &quot;unauthorized&quot; }
&lt;p&gt;The operation after which I see this warning is mostly:&lt;br/&gt;
  a) Wed Sep  5 07:04:21 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn847592&amp;#93;&lt;/span&gt; command history.$cmd command: { findandmodify: &quot;messages&quot;, query: { serverMessageId: &quot;some-value&quot;, $or: [ { att.r: &lt;/p&gt;
{ $exists: false }
&lt;p&gt; }, { att.r: &lt;/p&gt;
{ $type: 2 }
&lt;p&gt; } ] }, update: { $set: &lt;/p&gt;
{ att.r: new Date(1346828661274) }
&lt;p&gt; } } ntoreturn:1 reslen:737 21ms&lt;br/&gt;
  b) Wed Sep  5 07:05:36 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn847628&amp;#93;&lt;/span&gt; command history.$cmd command: { $eval: &quot;function(accounts, limitDate)&lt;/p&gt;
{ return magicContactMap(accounts, limitDate);}
&lt;p&gt;&quot;, args: [ [ &quot;k@domain.com&quot;, &quot;-6@chat.facebook.com~fb&quot;, &quot;x@gmail.com~gt&quot; ], new Date(1341644733695) ], nolock: true } ntoreturn:1 reslen:4703 2353ms &lt;/p&gt;

&lt;p&gt;Last 1.5 days, I got some 225 such warning messages. But as I read over the forum, its not much to worry about.&lt;/p&gt;

&lt;p&gt;There is a $eval process, as mentioned above. But again, this can&apos;t be the reason, because sometimes I faced the same issue with the secondary, which don&apos;t receive any reads(slaveOk = disabled).&lt;/p&gt;</comment>
                            <comment id="161792" author="renctan" created="Fri, 7 Sep 2012 21:10:47 +0000"  >&lt;p&gt;It would be helpful if you can provide the output of db.currentOp()? This operation does not require a lock, so you can possibly get some response from it, unless the server is overloaded. Do you also have these servers in MMS?&lt;/p&gt;

&lt;p&gt;Do you use server side javascript features like $where, $eval or mapreduce?&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>12.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 7 Sep 2012 21:10:47 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        11 years, 18 weeks, 3 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            11 years, 18 weeks, 3 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>abhishekiitg10@gmail.com</customfieldvalue>
            <customfieldvalue>randolph@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrnpk7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrgv1z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>11993</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht05mf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>