<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:11:05 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-68541] Concurrent removeShard and movePrimary may delete unsharded collections</title>
                <link>https://jira.mongodb.org/browse/SERVER-68541</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Concurrent &lt;tt&gt;removeShard&lt;/tt&gt; and &lt;tt&gt;movePrimary&lt;/tt&gt; may end up with an undesired delete of unsharded collections.&lt;/p&gt;

&lt;p&gt;&lt;ins&gt;Bug description&lt;/ins&gt;&lt;br/&gt;
Imagine the following scenario:&lt;/p&gt;

&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;There are 2 shards: &apos;&lt;tt&gt;shard0&lt;/tt&gt;&apos;, &apos;shard1&apos;&lt;/li&gt;
	&lt;li&gt;Database &apos;&lt;tt&gt;myDB&lt;/tt&gt;&apos; primary shard is &apos;shard0&apos;&lt;/li&gt;
	&lt;li&gt;Collection &apos;&lt;tt&gt;myDB.collA&lt;/tt&gt;&apos; is unsharded, so it&apos;s located in &apos;shard0&apos;&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;At some point, someone decides to call concurrently these commands:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;{ removeShard:&apos;shard1&apos; }&lt;/li&gt;
	&lt;li&gt;{ movePrimary:&apos;myDB&apos;, to: &apos;shard1&apos;}
&lt;p&gt;.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Then, if the sequence of the internal executions are the written below, the cluster will end up with an undesired deletion of all the unsharded collections of &apos;myDB&apos;.&lt;/p&gt;

&lt;p&gt;1. &lt;tt&gt;removeShard&lt;/tt&gt; command is called to the config server&lt;br/&gt;
2. The config server, following the &lt;tt&gt;removeShard&lt;/tt&gt; thread, checks if the unsharded databases count on the shard is zero. As it&apos;s true, the process continues.&lt;br/&gt;
3. After that point, the &lt;tt&gt;movePrimary&lt;/tt&gt; is performed, which means that all the unsharded collections are moved to &apos;shard1&apos;.&lt;br/&gt;
4. The &lt;tt&gt;removeShard&lt;/tt&gt; commit phase starts and &apos;shard1&apos; is removed from the topology of the cluster.&lt;/p&gt;

&lt;p&gt;&lt;ins&gt;Small note to understand better the 2nd bullet&lt;/ins&gt;: the &lt;tt&gt;removeShard&lt;/tt&gt; command returns a non completed status if the shard still have unsharded databases and notifies the user that those should be moved explicitly using &lt;tt&gt;movePrimary&lt;/tt&gt;. A better explanation can be found &lt;a href=&quot;https://www.mongodb.com/docs/manual/tutorial/remove-shards-from-cluster/#move-databases-to-another-primary-shard&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2105899">SERVER-68541</key>
            <summary>Concurrent removeShard and movePrimary may delete unsharded collections</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="antonio.fuschetto@mongodb.com">Antonio Fuschetto</assignee>
                                    <reporter username="silvia.surroca@mongodb.com">Silvia Surroca</reporter>
                        <labels>
                            <label>data-loss</label>
                    </labels>
                <created>Wed, 3 Aug 2022 17:20:56 +0000</created>
                <updated>Sun, 29 Oct 2023 21:34:58 +0000</updated>
                            <resolved>Wed, 31 Aug 2022 16:21:07 +0000</resolved>
                                                    <fixVersion>6.1.1</fixVersion>
                    <fixVersion>6.0.3</fixVersion>
                    <fixVersion>6.2.0-rc0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>9</watches>
                                                                                                                <comments>
                            <comment id="4964379" author="xgen-internal-githook" created="Tue, 8 Nov 2022 18:31:38 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Antonio Fuschetto&apos;, &apos;email&apos;: &apos;antonio.fuschetto@mongodb.com&apos;, &apos;username&apos;: &apos;afuschetto&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-68541&quot; title=&quot;Concurrent removeShard and movePrimary may delete unsharded collections&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-68541&quot;&gt;&lt;del&gt;SERVER-68541&lt;/del&gt;&lt;/a&gt; Serialize the removeShard and commitMovePrimary commands to prevent the loss of moved collections&lt;br/&gt;
Branch: v6.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/50e767b1dbbd5104959989e056d3bd04b6119748&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/50e767b1dbbd5104959989e056d3bd04b6119748&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4964168" author="xgen-internal-githook" created="Tue, 8 Nov 2022 17:41:35 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Antonio Fuschetto&apos;, &apos;email&apos;: &apos;antonio.fuschetto@mongodb.com&apos;, &apos;username&apos;: &apos;afuschetto&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-68541&quot; title=&quot;Concurrent removeShard and movePrimary may delete unsharded collections&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-68541&quot;&gt;&lt;del&gt;SERVER-68541&lt;/del&gt;&lt;/a&gt; Serialize the removeShard and commitMovePrimary commands to prevent the loss of moved collections&lt;br/&gt;
Branch: v6.1&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/eb90b652aa02c9f6b1d805abfd6956ba7b312f60&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/eb90b652aa02c9f6b1d805abfd6956ba7b312f60&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4784544" author="xgen-internal-githook" created="Mon, 29 Aug 2022 17:02:53 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Antonio Fuschetto&apos;, &apos;email&apos;: &apos;antonio.fuschetto@mongodb.com&apos;, &apos;username&apos;: &apos;afuschetto&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-68541&quot; title=&quot;Concurrent removeShard and movePrimary may delete unsharded collections&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-68541&quot;&gt;&lt;del&gt;SERVER-68541&lt;/del&gt;&lt;/a&gt; Concurrent removeShard and movePrimary may delete unsharded collections&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/cf84bec54627ba1efb6aebd829f6b3d11aaf112e&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/cf84bec54627ba1efb6aebd829f6b3d11aaf112e&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4783015" author="xgen-internal-githook" created="Mon, 29 Aug 2022 08:49:04 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Antonio Fuschetto&apos;, &apos;email&apos;: &apos;antonio.fuschetto@mongodb.com&apos;, &apos;username&apos;: &apos;afuschetto&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-68541&quot; title=&quot;Concurrent removeShard and movePrimary may delete unsharded collections&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-68541&quot;&gt;&lt;del&gt;SERVER-68541&lt;/del&gt;&lt;/a&gt; Concurrent removeShard and movePrimary may delete unsharded collections&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/107d9c38caae897b7e99af3db4ec429039936c87&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/107d9c38caae897b7e99af3db4ec429039936c87&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4736922" author="JIRAUSER1259062" created="Tue, 9 Aug 2022 07:47:33 +0000"  >&lt;h2&gt;&lt;a name=&quot;Proposedsolution&quot;&gt;&lt;/a&gt;Proposed solution&lt;/h2&gt;

&lt;p&gt;Following the logic currently implemented &lt;a href=&quot;https://github.com/10gen/mongo/blob/58703826eabe1351692fd0efa56d9d1665896b45/src/mongo/db/s/config/sharding_catalog_manager_chunk_operations.cpp#L981-L1247&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;to commit the chunk migration&lt;/a&gt;, it seems natural to adopt the same approach consisting in 1) to expose a new config server command (i.e., &lt;tt&gt;_configsvrCommitMovePrimary&lt;/tt&gt;) to atomically commit the configuration changes required by the &lt;tt&gt;movePrimary&lt;/tt&gt; command, and 2) to synchronize these configuration changes with the &lt;tt&gt;removeShard&lt;/tt&gt; command (reusing an existing mutex).&lt;/p&gt;

&lt;p&gt;This solution serializes the configuration changes of concurrent invocations of the &lt;tt&gt;removeShard&lt;/tt&gt; and &lt;tt&gt;movePrimary&lt;/tt&gt; commands and then resolves the bug in question.&lt;/p&gt;
&lt;h2&gt;&lt;a name=&quot;Backwardcompatibility&quot;&gt;&lt;/a&gt;Backward compatibility&lt;/h2&gt;

&lt;p&gt;Depending on the versions to which the fix needs to be back-ported (potentially all), the donor shard could fall back into the current logic (consisting in finding the current primary shard for the given database and then committing changes) if the new config server command is not exposed (e.g. in a multiversion deployment).&lt;/p&gt;

&lt;p&gt;Also, up to 5.0 version, &lt;a href=&quot;https://github.com/mongodb/mongo/blob/v5.0/src/mongo/s/request_types/move_primary.idl&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;the config server already exposes the &lt;tt&gt;_configsvrCommitMovePrimary&lt;/tt&gt; command&lt;/a&gt; and it would be interesting to understand why the logic was changed. Was the goal to decentralize the configuration server logic? However, the idea (to be validated) is to use the same command to have a compatible solution with 5.0 version and lower.&lt;/p&gt;</comment>
                            <comment id="4728060" author="kaloian.manassiev" created="Thu, 4 Aug 2022 09:53:57 +0000"  >&lt;p&gt;Hmm, will this actually get fixed just by the movePrimary coordinator implementation by itself? There is nothing that prevents even at commit time of the shard removal that the commit of the new placement will not happen &lt;b&gt;after&lt;/b&gt; the shard has been removed.&lt;/p&gt;

&lt;p&gt;I don&apos;t think we need to wait until the Add/Remove Shard project, but the move primary commit needs to become a command on the CSRS which serialises with the shard removal lock. CC &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=antonio.fuschetto%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;antonio.fuschetto@mongodb.com&quot;&gt;antonio.fuschetto@mongodb.com&lt;/a&gt; to keep in mind.&lt;/p&gt;</comment>
                            <comment id="4728032" author="JIRAUSER1263826" created="Thu, 4 Aug 2022 09:46:26 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev@mongodb.com&quot;&gt;kaloian.manassiev@mongodb.com&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=antonio.fuschetto%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;antonio.fuschetto@mongodb.com&quot;&gt;antonio.fuschetto@mongodb.com&lt;/a&gt; Considering as part of Sharding First we are going to rewrite the MovePrimary coordinator, we should also fix this one. &lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="2142967">SERVER-69890</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="393867" name="repro-undesired-unsharded-collections-remove.patch" size="4788" author="silvia.surroca@mongodb.com" created="Wed, 3 Aug 2022 16:49:07 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="24444"><![CDATA[v6.1]]></customfieldvalue>
    <customfieldvalue key="23470"><![CDATA[v6.0]]></customfieldvalue>
    <customfieldvalue key="21777"><![CDATA[v5.0]]></customfieldvalue>
    <customfieldvalue key="18953"><![CDATA[v4.4]]></customfieldvalue>
    <customfieldvalue key="16775"><![CDATA[v4.2]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 4 Aug 2022 09:46:26 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 13 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 13 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>antonio.fuschetto@mongodb.com</customfieldvalue>
            <customfieldvalue>cris.insignarescuello@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
            <customfieldvalue>silvia.surroca@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i14ytb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i0nrk8:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="6395">Sharding EMEA 2022-08-22</customfieldvalue>
    <customfieldvalue id="6396">Sharding EMEA 2022-09-05</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;&lt;span class=&quot;nobr&quot;&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/attachment/393867/393867_repro-undesired-unsharded-collections-remove.patch&quot; title=&quot;repro-undesired-unsharded-collections-remove.patch attached to SERVER-68541&quot;&gt;repro-undesired-unsharded-collections-remove.patch&lt;sup&gt;&lt;img class=&quot;rendericon&quot; src=&quot;https://jira.mongodb.org/images/icons/link_attachment_7.gif&quot; height=&quot;7&quot; width=&quot;7&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;br/&gt;
Apply the provided patch on top of commit &lt;tt&gt;r6.1.0-alpha-1938-gfe099ee11c9&lt;/tt&gt; and run &lt;tt&gt;jstests/sharding/remove_shard_and_move_primary.js&lt;/tt&gt; in the &lt;tt&gt;sharding&lt;/tt&gt; suite.&lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i14kyn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>