<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:01:33 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-2918] Server shuts down once a day, without being able to reproduce the crash</title>
                <link>https://jira.mongodb.org/browse/SERVER-2918</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We&apos;ve been having problems with mongodb since updating to 1.6.5 (about 1 month ago). It is a single server configuration - we have 7 GB in (data + indexes) and mongodb usually stays at around 7-8 GB RAM, but the machine has 15 GB RAM in total and 4 GB swap.&lt;/p&gt;

&lt;p&gt;The database is crashing every day with two different patterns:&lt;br/&gt;
1) Mongodb fills the whole memory on the server (all the 15 GB and the whole 4 GB swap) very fast (so it goes from 7-8 GB to 15 + 4 in about a minute); after that, nothing seems to be able to shut the server down (we tried db.shutdownServer(), kill -15, kill -9) - it&apos;s blocked, it accepts new connections, but doesn&apos;t return anything; the only shut down procedure that seems to work is shutting the whole machine down. After restarting the machine, the lock file is present so mongodb won&apos;t start, but after removing the lock file, the server starts and it&apos;s ok again (for a day or two).&lt;br/&gt;
2) The second pattern is not present every day, more like once every 5 days - the database just shuts down, freeing all the memory. Since I just observed this problem, I can isolate the connection that seems to have caused the shutdown. Here is how the log looks like, for this connection:&lt;/p&gt;

&lt;p&gt;Fri Apr  8 10:37:14 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; run command jbm.$cmd { mapreduce: &quot;mongocvs&quot;, map: CodeWScope( function() { var x; var folders = this.employers&lt;span class=&quot;error&quot;&gt;&amp;#91;96012&amp;#93;&lt;/span&gt;.folders; for(x in folders) &lt;/p&gt;
{emit(x,1);}
&lt;p&gt; }, {}), reduce: CodeWScope( function(k, vals) { var sum = 0; for (var i in vals) &lt;/p&gt;
{ sum += vals[i]; }
&lt;p&gt; return sum; }, {}), query: { empids: 96012, $or: [ &lt;/p&gt;
{ folders: 815493 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 814868 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 813880 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811648 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811647 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811644 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811643 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811642 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811640 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811639 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811638 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811637 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 806672 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 806671 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 806670 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 806583 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 806493 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 805036 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 805035 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 801933 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 801932 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 801929 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 801648 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 800608 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 798096 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 797344 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 797306 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 796627 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 796626 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 796625 }
&lt;p&gt; ] }, out: &quot;mr_count_folder_cvs&quot; }&lt;br/&gt;
Fri Apr  8 10:37:14 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; mr ns: jbm.mongocvs&lt;br/&gt;
Fri Apr  8 10:37:14 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; run command jbm.$cmd { count: &quot;system.namespaces&quot;, query: &lt;/p&gt;
{ name: &quot;jbm.mongocvs&quot; }
&lt;p&gt; }&lt;br/&gt;
Fri Apr  8 10:37:14 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; query jbm.$cmd ntoreturn:1 command: { count: &quot;system.namespaces&quot;, query: &lt;/p&gt;
{ name: &quot;jbm.mongocvs&quot; }
&lt;p&gt; } reslen:64 0ms&lt;br/&gt;
Fri Apr  8 10:37:14 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; query jbm.system.js reslen:36 nreturned:0 0ms&lt;br/&gt;
Fri Apr  8 10:37:14 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; run command jbm.$cmd &lt;/p&gt;
{ drop: &quot;tmp.mr.mapreduce_1302248234_47440&quot; }
&lt;p&gt;Fri Apr  8 10:37:14 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; CMD: drop jbm.tmp.mr.mapreduce_1302248234_47440&lt;br/&gt;
Fri Apr  8 10:37:14 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; query jbm.$cmd ntoreturn:1 command: &lt;/p&gt;
{ drop: &quot;tmp.mr.mapreduce_1302248234_47440&quot; }
&lt;p&gt; reslen:78 0ms&lt;br/&gt;
Fri Apr  8 10:37:14 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; run command jbm.$cmd &lt;/p&gt;
{ drop: &quot;tmp.mr.mapreduce_1302248234_47440_inc&quot; }
&lt;p&gt;Fri Apr  8 10:37:14 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; CMD: drop jbm.tmp.mr.mapreduce_1302248234_47440_inc&lt;br/&gt;
Fri Apr  8 10:37:14 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; query jbm.$cmd ntoreturn:1 command: &lt;/p&gt;
{ drop: &quot;tmp.mr.mapreduce_1302248234_47440_inc&quot; }
&lt;p&gt; reslen:78 0ms&lt;br/&gt;
Fri Apr  8 10:37:14 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; create collection jbm.tmp.mr.mapreduce_1302248234_47440_inc &lt;/p&gt;
{ autoIndexId: 0 }
&lt;p&gt;Fri Apr  8 10:55:32 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; mr failed, removing collection&lt;br/&gt;
Fri Apr  8 10:55:32 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; run command jbm.$cmd &lt;/p&gt;
{ drop: &quot;tmp.mr.mapreduce_1302248234_47440&quot; }
&lt;p&gt;Fri Apr  8 10:57:01 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; CMD: drop jbm.tmp.mr.mapreduce_1302248234_47440&lt;br/&gt;
Fri Apr  8 10:57:01 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; query jbm.$cmd ntoreturn:1 command: &lt;/p&gt;
{ drop: &quot;tmp.mr.mapreduce_1302248234_47440&quot; }
&lt;p&gt; reslen:78 88937ms&lt;br/&gt;
Fri Apr  8 10:57:01 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; run command jbm.$cmd &lt;/p&gt;
{ drop: &quot;tmp.mr.mapreduce_1302248234_47440_inc&quot; }
&lt;p&gt;Fri Apr  8 10:57:03 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; CMD: drop jbm.tmp.mr.mapreduce_1302248234_47440_inc&lt;br/&gt;
Fri Apr  8 10:57:03 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; dropCollection: jbm.tmp.mr.mapreduce_1302248234_47440_inc&lt;br/&gt;
Fri Apr  8 10:57:03 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt;          dropIndexes done&lt;br/&gt;
Fri Apr  8 10:57:03 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; query jbm.$cmd ntoreturn:1 command: &lt;/p&gt;
{ drop: &quot;tmp.mr.mapreduce_1302248234_47440_inc&quot; }
&lt;p&gt; reslen:105 2710ms&lt;br/&gt;
Fri Apr  8 10:57:04 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt;   Uncaught std::exception: St9bad_alloc, terminating&lt;br/&gt;
Fri Apr  8 10:57:04 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; shutdown: going to close listening sockets...&lt;br/&gt;
Fri Apr  8 10:57:04 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; closing listening socket: 5&lt;br/&gt;
Fri Apr  8 10:57:04 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; closing listening socket: 6&lt;br/&gt;
Fri Apr  8 10:57:04 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; closing listening socket: 7&lt;br/&gt;
Fri Apr  8 10:57:04 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; closing listening socket: 8&lt;br/&gt;
Fri Apr  8 10:57:04 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; shutdown: going to flush oplog...&lt;br/&gt;
Fri Apr  8 10:57:04 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; shutdown: going to close sockets...&lt;br/&gt;
Fri Apr  8 10:57:04 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; shutdown: waiting for fs preallocator...&lt;br/&gt;
Fri Apr  8 10:57:04 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; shutdown: closing all files...&lt;br/&gt;
Fri Apr  8 10:57:04 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn58463&amp;#93;&lt;/span&gt; shutdown: removing fs lock...&lt;/p&gt;

&lt;p&gt;Our plans for the immediate future are to upgrade to 1.8 version and start another machine, transforming the system into a replica set - and that is what we are working at right now. But I still want to investigate this problem and be sure that it won&apos;t be present after the upgrade. &lt;/p&gt;

&lt;p&gt;Notes:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;unfortunately the shutdown I just described is not reproducible - I tried the same sequence of commands and it didn&apos;t crash&lt;/li&gt;
	&lt;li&gt;as you can see from the log, we use map-reduce too&lt;/li&gt;
&lt;/ul&gt;
</description>
                <environment></environment>
        <key id="15780">SERVER-2918</key>
            <summary>Server shuts down once a day, without being able to reproduce the crash</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="catalint">Catalin Trandafir</reporter>
                        <labels>
                    </labels>
                <created>Fri, 8 Apr 2011 11:10:14 +0000</created>
                <updated>Tue, 12 Jul 2016 00:18:03 +0000</updated>
                            <resolved>Fri, 8 Apr 2011 16:04:43 +0000</resolved>
                                    <version>1.6.5</version>
                                                    <component>Stability</component>
                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="28345" author="eliot" created="Fri, 8 Apr 2011 16:04:43 +0000"  >&lt;p&gt;There were a couple of memory leaks in JS that can cause issues with spidermonkey, which is why its happening in 1.6.5&lt;/p&gt;</comment>
                            <comment id="28332" author="catalint" created="Fri, 8 Apr 2011 14:29:12 +0000"  >&lt;p&gt;Thank you for the loop idea - at least I was able to reproduce the crash. I still don&apos;t understand why it is crashing on 1.6.5, but being able to reproduce it makes it a lot less frustrating.&lt;/p&gt;</comment>
                            <comment id="28331" author="catalint" created="Fri, 8 Apr 2011 14:26:12 +0000"  >&lt;p&gt;I tried to run it in a loop:&lt;/p&gt;

&lt;p&gt;m = function() { var x; var folders = this.employers&lt;span class=&quot;error&quot;&gt;&amp;#91;96012&amp;#93;&lt;/span&gt;.folders; for(x in folders) &lt;/p&gt;
{emit(x,1);}
&lt;p&gt; };&lt;/p&gt;

&lt;p&gt;r = function(k, vals) { var sum = 0; for (var i in vals) &lt;/p&gt;
{ sum += vals[i]; }
&lt;p&gt; return sum; };&lt;/p&gt;

&lt;p&gt;for ( i=1; i&amp;lt;1000; i++ ){&lt;br/&gt;
	res = db.runCommand({&lt;br/&gt;
		mapreduce: &quot;mongocvs&quot;,&lt;br/&gt;
		map: m,&lt;br/&gt;
		reduce: r,&lt;br/&gt;
		query: { empids: 96012, $or: [ &lt;/p&gt;
{ folders: 815493 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 814868 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 813880 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811648 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811647 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811644 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811643 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811642 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811640 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811639 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811638 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 811637 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 806672 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 806671 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 806670 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 806583 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 806493 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 805036 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 805035 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 801933 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 801932 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 801929 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 801648 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 800608 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 798096 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 797344 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 797306 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 796627 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 796626 }
&lt;p&gt;, &lt;/p&gt;
{ folders: 796625 }
&lt;p&gt; ] },&lt;br/&gt;
		out: &quot;mr_count_folder_cvs&quot;&lt;br/&gt;
	});&lt;br/&gt;
}&lt;/p&gt;



&lt;p&gt;I ran this in a test environment, on 1.6.5 version - it crashed the server (tried it twice); then I tried this on another server, with 1.8 version, same data, replicaset environment, but only with the primary started (and 2 arbiters) - on 1.8 version the server doesn&apos;t crash.&lt;/p&gt;</comment>
                            <comment id="28328" author="eliot" created="Fri, 8 Apr 2011 12:30:47 +0000"  >&lt;p&gt;Looks like a memory leak, possible in map/reduce.  If you run that same command in a loop, does it crash?&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 8 Apr 2011 12:30:47 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        12 years, 45 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            12 years, 45 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>catalint</customfieldvalue>
            <customfieldvalue>eliot</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrp1nj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hries7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>21047</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht0c2v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>