<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:01:59 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-3067] Can&apos;t kill indexing operations</title>
                <link>https://jira.mongodb.org/browse/SERVER-3067</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Overview:&lt;/p&gt;

&lt;p&gt;This ticket adds support for using killop to interrupt a client initiated foreground index build in progress.  When the index build is killed, all resources related to the index are cleaned up and the index is removed from system.indexes.  An error response is provided on the connection that initiated the index build.  A foreground index build will similarly be killed if mongod is shut down while the build is in progress.&lt;/p&gt;

&lt;p&gt;Index builds that are not directly initiated by an external client cannot be interrupted in this manner.  For example, map reduce and reindex (as well as other commands) build indexes as part of their internal implementations and cannot be interrupted currently.&lt;/p&gt;

&lt;p&gt;Aaron&lt;/p&gt;

&lt;p&gt;----------------------&lt;/p&gt;


&lt;p&gt;I accidentally started a foreground index build on 300 million records. I followed the documentation and tried to kill it, but was unsuccessful. This resulted in significant downtime.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;http://www.mongodb.org/display/DOCS/Viewing+and+Terminating+Current+Operation&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://www.mongodb.org/display/DOCS/Viewing+and+Terminating+Current+Operation&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;{&lt;br/&gt;
       &quot;opid&quot; : 710799555,&lt;br/&gt;
       &quot;active&quot; : true,&lt;br/&gt;
       &quot;lockType&quot; : &quot;write&quot;,&lt;br/&gt;
       &quot;waitingForLock&quot; : false,&lt;br/&gt;
       &quot;secs_running&quot; : 5260,&lt;br/&gt;
       &quot;op&quot; : &quot;insert&quot;,&lt;br/&gt;
       &quot;ns&quot; : &quot;production.system.indexes&quot;,&lt;br/&gt;
       &quot;query&quot; : {&lt;/p&gt;

&lt;p&gt;       },&lt;br/&gt;
       &quot;client&quot; : &quot;10.122.166.155:57729&quot;,&lt;br/&gt;
       &quot;desc&quot; : &quot;conn&quot;,&lt;br/&gt;
       &quot;msg&quot; : &quot;index: (2/3) btree bottom up 192158787/298486401 64%&quot;&lt;br/&gt;
}&lt;/p&gt;

&lt;p&gt;I issued the command:&lt;br/&gt;
db.killOp(710799555)&lt;/p&gt;

&lt;p&gt;The console printed out that it was trying to kill the process, but it never did. Once the master finally completed, after over an hour, we had to wait for the replica set slaves to also build the index. The overall outage window lasted several hours but could have been avoided if I could kill the job.&lt;/p&gt;</description>
                <environment>Ubuntu 10.04.2&lt;br/&gt;
Linux 2.6.32-30-generic #59-Ubuntu SMP Tue Mar 1 21:30:46 UTC 2011 x86_64 GNU/Linux</environment>
        <key id="16793">SERVER-3067</key>
            <summary>Can&apos;t kill indexing operations</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="aaron">Aaron Staple</assignee>
                                    <reporter username="aaron.westendorf">Aaron Westendorf</reporter>
                        <labels>
                    </labels>
                <created>Mon, 9 May 2011 16:31:17 +0000</created>
                <updated>Wed, 28 Oct 2015 04:26:20 +0000</updated>
                            <resolved>Fri, 9 Nov 2012 17:18:02 +0000</resolved>
                                    <version>1.8.0</version>
                                    <fixVersion>2.3.1</fixVersion>
                                    <component>Index Maintenance</component>
                                        <votes>12</votes>
                                    <watches>25</watches>
                                                                                                                <comments>
                            <comment id="458365" author="jalava" created="Tue, 19 Nov 2013 11:00:13 +0000"  >&lt;p&gt;Just happened to browse by this issue, and had to pitch in.&lt;/p&gt;

&lt;p&gt;We killed background indexing task about month ago and this corrupted the database with invalid BSONSize error. Cannot find any logs about that anymore, but just a note that it can happen.&lt;/p&gt;

&lt;p&gt;Collection already had existing sparse &amp;amp; unique indexes, and this was an additional index to the collection.&lt;/p&gt;</comment>
                            <comment id="184427" author="auto" created="Fri, 9 Nov 2012 06:15:33 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;date&apos;: u&apos;2012-11-09T04:51:57Z&apos;, u&apos;email&apos;: u&apos;aaron@10gen.com&apos;, u&apos;name&apos;: u&apos;Aaron&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3067&quot; title=&quot;Can&amp;#39;t kill indexing operations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3067&quot;&gt;&lt;del&gt;SERVER-3067&lt;/del&gt;&lt;/a&gt; Disable BSONObjExternalSorter&apos;s qsort callback interrupts on solaris.  Our solaris build environment does not support exceptions in qsort callbacks.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/bdd827db775d7f05de50908e45aca93e0b5c26b6&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/bdd827db775d7f05de50908e45aca93e0b5c26b6&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="184397" author="auto" created="Fri, 9 Nov 2012 03:16:13 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;date&apos;: u&apos;2012-11-09T03:15:03Z&apos;, u&apos;email&apos;: u&apos;aaron@10gen.com&apos;, u&apos;name&apos;: u&apos;Aaron&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3067&quot; title=&quot;Can&amp;#39;t kill indexing operations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3067&quot;&gt;&lt;del&gt;SERVER-3067&lt;/del&gt;&lt;/a&gt; Fix windows compile by specifying boost::shared_ptr.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/75aef9e0f2a1c8d825f280b1210a24a4d23aabf7&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/75aef9e0f2a1c8d825f280b1210a24a4d23aabf7&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="184394" author="auto" created="Fri, 9 Nov 2012 03:03:40 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;date&apos;: u&apos;2012-11-09T02:51:33Z&apos;, u&apos;email&apos;: u&apos;aaron@10gen.com&apos;, u&apos;name&apos;: u&apos;Aaron&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3067&quot; title=&quot;Can&amp;#39;t kill indexing operations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3067&quot;&gt;&lt;del&gt;SERVER-3067&lt;/del&gt;&lt;/a&gt; Fix rhel warnings.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/f225b710cf39e462f782e14d4b517596226f161a&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/f225b710cf39e462f782e14d4b517596226f161a&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="184393" author="auto" created="Fri, 9 Nov 2012 03:03:38 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;date&apos;: u&apos;2012-11-09T01:19:32Z&apos;, u&apos;email&apos;: u&apos;aaron@10gen.com&apos;, u&apos;name&apos;: u&apos;Aaron&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3067&quot; title=&quot;Can&amp;#39;t kill indexing operations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3067&quot;&gt;&lt;del&gt;SERVER-3067&lt;/del&gt;&lt;/a&gt; Move old external sort tests from jsobjtests.cpp to extsorttests.cpp&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/5509b014eb5700543f0339240319f2ebac3bbbe9&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/5509b014eb5700543f0339240319f2ebac3bbbe9&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="184350" author="auto" created="Fri, 9 Nov 2012 00:32:49 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;date&apos;: u&apos;2012-10-25T21:07:23Z&apos;, u&apos;email&apos;: u&apos;aaron@10gen.com&apos;, u&apos;name&apos;: u&apos;Aaron&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3067&quot; title=&quot;Can&amp;#39;t kill indexing operations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3067&quot;&gt;&lt;del&gt;SERVER-3067&lt;/del&gt;&lt;/a&gt; Add killop support for foreground index builds.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/6a51b6b01e4ebdd723e6ad33f07934d5558f9ad7&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/6a51b6b01e4ebdd723e6ad33f07934d5558f9ad7&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="178714" author="auto" created="Wed, 24 Oct 2012 16:05:21 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;date&apos;: u&apos;2012-10-23T18:19:29-07:00&apos;, u&apos;email&apos;: u&apos;aaron@10gen.com&apos;, u&apos;name&apos;: u&apos;Aaron&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3067&quot; title=&quot;Can&amp;#39;t kill indexing operations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3067&quot;&gt;&lt;del&gt;SERVER-3067&lt;/del&gt;&lt;/a&gt; Remove tab characters from the btreebuilder.cpp file.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/df9bc9ca6193fd028045637360ed2dd4aff760e0&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/df9bc9ca6193fd028045637360ed2dd4aff760e0&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="152151" author="vinaykr" created="Thu, 9 Aug 2012 23:34:07 +0000"  >&lt;p&gt;We had a similar issue where foreground index creation froze the entire cluster and starved all other read/write ops. It should not be so easy to get into this stage. The default option for index creation should be changed to &quot;background always&quot; because for any reasonably sized production deployment, foreground index creation never makes any sense. If it&apos;s a small sized cluster/db then even background index creation will complete quickly. So, I think the default doesnt make sense. &lt;br/&gt;
Till that setting is changed, at least index operations should allowed to be killed otherwise you are just screwed!&lt;br/&gt;
Thanks!&lt;/p&gt;</comment>
                            <comment id="144535" author="colinhowe" created="Wed, 18 Jul 2012 16:49:38 +0000"  >&lt;p&gt;Hi Ian,&lt;/p&gt;

&lt;p&gt;I&apos;d like to stress how important this is for us. We can have all the replication under the sun, all the multi-DC writes you can shake a stick at... but it is all for nought if I can accidentally lock out an entire replication set by an accidental omission of background: true or a migration accidentally not performed in office hours... &lt;/p&gt;

&lt;p&gt;The fact it then also replicates the index build command to replicas just makes this even worse.&lt;/p&gt;

&lt;p&gt;This, or the suggestion of preventing foreground index builds, would go a long way to making us believe that MongoDB is a robust solution.&lt;/p&gt;


&lt;p&gt;Thanks,&lt;br/&gt;
Colin&lt;/p&gt;</comment>
                            <comment id="144130" author="ian@10gen.com" created="Tue, 17 Jul 2012 17:34:36 +0000"  >&lt;p&gt;Hi all, we&apos;re still in the planning stages for 2.4 and will evaluate getting this in, although I can&apos;t guarantee it will make it.&lt;/p&gt;</comment>
                            <comment id="144083" author="bartzy" created="Tue, 17 Jul 2012 16:39:17 +0000"  >&lt;p&gt;Are there any updates from 10gen people about this?&lt;/p&gt;

&lt;p&gt;Thanks!&lt;/p&gt;</comment>
                            <comment id="118853" author="bcambel@gmail.com" created="Sun, 13 May 2012 01:03:56 +0000"  >&lt;p&gt;db version v2.0.5&lt;/p&gt;

&lt;p&gt;I&apos;ve even realized that you can not also shutdown server to kill the indexing operation.&lt;/p&gt;


&lt;p&gt;db.shutdownServer()&lt;br/&gt;
assert failed : unexpected error: &quot;shutdownServer failed: db assertion failure&quot;&lt;br/&gt;
Error(&quot;Printing Stack Trace&quot;)@:0&lt;br/&gt;
()@shell/utils.js:35&lt;br/&gt;
(&quot;assert failed : unexpected error: \&quot;shutdownServer failed: db assertion failure\&quot;&quot;)@shell/utils.js:46&lt;br/&gt;
(false,&quot;unexpected error: \&quot;shutdownServer failed: db assertion failure\&quot;&quot;)@shell/utils.js:54&lt;br/&gt;
()@shell/db.js:205&lt;br/&gt;
@(shell):1&lt;/p&gt;

&lt;p&gt;Sun May 13 00:57:58 uncaught exception: assert failed : unexpected error: &quot;shutdownServer failed: db assertion failure&quot;&lt;/p&gt;</comment>
                            <comment id="115965" author="calsaverini" created="Thu, 3 May 2012 13:04:38 +0000"  >&lt;p&gt;I&apos;ve got the same problem in 1.8.2. Luckily it wasn&apos;t in a production environment, but it blocked my work for a couple hours. &lt;/p&gt;

&lt;p&gt;I have two doubts:&lt;/p&gt;

&lt;p&gt;1) Is this corrected in newer versions?&lt;br/&gt;
2) If I just kill mongod while it&apos;s creating a new index, the collection may be corrupted?&lt;/p&gt;</comment>
                            <comment id="99363" author="colinhowe" created="Thu, 15 Mar 2012 20:38:44 +0000"  >&lt;p&gt;Hey, getting this in would be amazing. It&apos;s an absolute killer if someone accidentally (in code or manually) creates a foreground index and brings down an entire site without any option apart from a failover..&lt;/p&gt;</comment>
                            <comment id="94073" author="zacwitte" created="Thu, 1 Mar 2012 10:41:43 +0000"  >&lt;p&gt;Any updates to this? I still have the problem in 2.0.2&lt;/p&gt;

&lt;p&gt;Thu Mar  1 10:33:18 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; connection accepted from 127.0.0.1:38442 #3&lt;br/&gt;
Thu Mar  1 10:33:19 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn3&amp;#93;&lt;/span&gt; going to kill op: op: 89&lt;br/&gt;
Thu Mar  1 10:33:19 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn3&amp;#93;&lt;/span&gt; end connection 127.0.0.1:38442&lt;br/&gt;
                42431700/967940554      4%&lt;br/&gt;
                43073900/967940554      4%&lt;br/&gt;
                44016800/967940554      4%&lt;br/&gt;
                45115100/967940554      4%&lt;br/&gt;
                46394100/967940554      4%&lt;br/&gt;
                47535100/967940554      4%&lt;br/&gt;
                48772000/967940554      5%&lt;br/&gt;
...&lt;/p&gt;</comment>
                            <comment id="32952" author="aaron.westendorf" created="Mon, 16 May 2011 18:29:02 +0000"  >&lt;p&gt;I tested background indexing and found the following behavior:&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;While indexing, I could not run queries on the collection being indexed, though I could run other queries&lt;/li&gt;
	&lt;li&gt;I was able to kill the indexing job&lt;/li&gt;
	&lt;li&gt;After killing the job, queries to the collection that I had previously cancelled showed up in the process list&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;From this, I think it&apos;s the case that the lock on collections which an index grabs precludes other operations from running, possibly including the kill operation.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="24626">SERVER-4227</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="103227">SERVER-12164</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>16.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 11 May 2011 17:25:06 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        10 years, 13 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-4227'>SERVER-4227</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>dan@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            10 years, 13 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>aaron</customfieldvalue>
            <customfieldvalue>aaron.westendorf</customfieldvalue>
            <customfieldvalue>auto</customfieldvalue>
            <customfieldvalue>bcambel@gmail.com</customfieldvalue>
            <customfieldvalue>bartzy</customfieldvalue>
            <customfieldvalue>colinhowe</customfieldvalue>
            <customfieldvalue>ian@mongodb.com</customfieldvalue>
            <customfieldvalue>jalava</customfieldvalue>
            <customfieldvalue>calsaverini</customfieldvalue>
            <customfieldvalue>vinaykr</customfieldvalue>
            <customfieldvalue>zacwitte</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrp053:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrfx47:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5704</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10166" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Tests Written</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10154"><![CDATA[Complete]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrnv33:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>