<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:35:48 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-77513] Range deletion document removal commit must trigger in-memory state update</title>
                <link>https://jira.mongodb.org/browse/SERVER-77513</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Full explanation in final comment of BF-27927.&lt;/p&gt;

&lt;p&gt;The range deleter&apos;s onInserts op observer &lt;a href=&quot;https://github.com/mongodb/mongo/blob/3d63e44eb5f871bd0e3f193f80088f134589a157/src/mongo/db/s/range_deleter_service_op_observer.cpp#L50&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;schedules the update to happen on commit&lt;/a&gt; whereas the onDeletes op observer &lt;a href=&quot;https://github.com/mongodb/mongo/blob/3d63e44eb5f871bd0e3f193f80088f134589a157/src/mongo/db/s/range_deleter_service_op_observer.cpp#L141-L161&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;does the work right away&lt;/a&gt;. This is usually fine because there is sufficient time between inserting the range deletion document and deleting it that these do not overlap. However, in the presence of renames, the following scenario can occur:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;Migration finishes and a range deletion task is marked as ready for processing, thus scheduling it on the range deleter&lt;/li&gt;
	&lt;li&gt;The rename snapshots the range deletion task&lt;/li&gt;
	&lt;li&gt;The range deletion happens and the deletions are replicated to secondaries (but the task document is still there)&lt;/li&gt;
	&lt;li&gt;The rename inserts the snapshotted document into config.rangeDeletions&lt;/li&gt;
	&lt;li&gt;The range deleter deletes all overlapping task documents for the range&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;In this case, the insertion by the rename thread of the document and the deletion of that document can happen immediately after one another (within less than a millisecond). In this case, the updates to the range deleter&apos;s in memory state can be ordered incorrectly as follows:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;The onInserts observer runs, scheduling the work of inserting the range deletion in memory for the onCommit observer&lt;/li&gt;
	&lt;li&gt;The onDelete observer runs, removing the range deletion from in memory (which is a noop because it has not yet been added)&lt;/li&gt;
	&lt;li&gt;The onCommit observer runs, inserting the range deletion task into memory&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;The range deleter will then do the deletion for the in-memory range deletion, but there is no task document to remove and so the in memory state will never be cleared (since it is only updated via onDelete).&lt;/p&gt;

&lt;p&gt;In this scenario, we now have in-memory state for a range deletion that will never be removed. This prevents all further range deletions for this range because they will join the existing in memory registration.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&lt;del&gt;A possible solution to this would be to register the range deletion task in the onInserts observer rather than scheduling it for onCommit.&lt;/del&gt;&lt;/p&gt;</description>
                <environment></environment>
        <key id="2351540">SERVER-77513</key>
            <summary>Range deletion document removal commit must trigger in-memory state update</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="pierlauro.sciarelli@mongodb.com">Pierlauro Sciarelli</assignee>
                                    <reporter username="allison.easton@mongodb.com">Allison Easton</reporter>
                        <labels>
                            <label>auto-reverted</label>
                    </labels>
                <created>Fri, 26 May 2023 13:22:02 +0000</created>
                <updated>Mon, 20 Nov 2023 16:24:24 +0000</updated>
                            <resolved>Tue, 4 Jul 2023 07:11:04 +0000</resolved>
                                    <version>6.2.1</version>
                    <version>7.1.0-rc0</version>
                    <version>6.3.1</version>
                    <version>7.0.0-rc2</version>
                                    <fixVersion>7.1.0-rc0</fixVersion>
                    <fixVersion>7.0.0-rc7</fixVersion>
                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="5546698" author="xgen-internal-githook" created="Thu, 6 Jul 2023 10:11:00 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;Pierlauro Sciarelli&apos;, &apos;email&apos;: &apos;pierlauro.sciarelli@mongodb.com&apos;, &apos;username&apos;: &apos;pierlauro&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-77513&quot; title=&quot;Range deletion document removal commit must trigger in-memory state update&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-77513&quot;&gt;&lt;del&gt;SERVER-77513&lt;/del&gt;&lt;/a&gt; Range deletion document removal commit must trigger in-memory state update&lt;br/&gt;
Branch: v7.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/8aec6c6967ef6bbe5c1101bf7248d27bec0e647f&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/8aec6c6967ef6bbe5c1101bf7248d27bec0e647f&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5540066" author="xgen-internal-githook" created="Mon, 3 Jul 2023 10:07:46 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;Pierlauro Sciarelli&apos;, &apos;email&apos;: &apos;pierlauro.sciarelli@mongodb.com&apos;, &apos;username&apos;: &apos;pierlauro&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-77513&quot; title=&quot;Range deletion document removal commit must trigger in-memory state update&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-77513&quot;&gt;&lt;del&gt;SERVER-77513&lt;/del&gt;&lt;/a&gt; Range deletion document removal commit must trigger in-memory state update&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/96d69211424ea908eb3447999d0e624150efee4d&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/96d69211424ea908eb3447999d0e624150efee4d&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5539573" author="xgen-internal-githook" created="Mon, 3 Jul 2023 00:08:17 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;auto-revert-processor&apos;, &apos;email&apos;: &apos;dev-prod-dag@mongodb.com&apos;, &apos;username&apos;: &apos;&apos;}
&lt;p&gt;Message: Revert &quot;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-77513&quot; title=&quot;Range deletion document removal commit must trigger in-memory state update&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-77513&quot;&gt;&lt;del&gt;SERVER-77513&lt;/del&gt;&lt;/a&gt; Range deletion document removal commit must trigger in-memory state update&quot;&lt;/p&gt;

&lt;p&gt;This reverts commit 866f81bb3016d4db13771a2be827f17cb6520ea1.&lt;br/&gt;
Branch: EVG-17874-taskgen-test&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/482bf3ab3660f0124a8bf5fba7d3fbeaba8c51a6&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/482bf3ab3660f0124a8bf5fba7d3fbeaba8c51a6&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5539554" author="xgen-internal-githook" created="Mon, 3 Jul 2023 00:07:31 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;Pierlauro Sciarelli&apos;, &apos;email&apos;: &apos;pierlauro.sciarelli@mongodb.com&apos;, &apos;username&apos;: &apos;pierlauro&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-77513&quot; title=&quot;Range deletion document removal commit must trigger in-memory state update&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-77513&quot;&gt;&lt;del&gt;SERVER-77513&lt;/del&gt;&lt;/a&gt; Range deletion document removal commit must trigger in-memory state update&lt;br/&gt;
Branch: EVG-17874-taskgen-test&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/866f81bb3016d4db13771a2be827f17cb6520ea1&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/866f81bb3016d4db13771a2be827f17cb6520ea1&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5538401" author="xgen-buildbaron-user" created="Sat, 1 Jul 2023 01:55:30 +0000"  >&lt;p&gt;Ticket re-opened due to revert. &lt;a href=&quot;https://evergreen.mongodb.com/task/mongodb_mongo_master_enterprise_rhel_80_64_bit_dynamic_display_sharding_csrs_continuous_config_stepdown_866f81bb3016d4db13771a2be827f17cb6520ea1_23_06_30_13_02_48&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;sharding_csrs_continuous_config_stepdown&lt;/a&gt; began a consistent failure of jstests/sharding/analyze_shard_key/analyze_shard_key_read_preference.js&lt;/p&gt;</comment>
                            <comment id="5538399" author="xgen-internal-githook" created="Sat, 1 Jul 2023 01:54:44 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;auto-revert-processor&apos;, &apos;email&apos;: &apos;dev-prod-dag@mongodb.com&apos;, &apos;username&apos;: &apos;&apos;}
&lt;p&gt;Message: Revert &quot;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-77513&quot; title=&quot;Range deletion document removal commit must trigger in-memory state update&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-77513&quot;&gt;&lt;del&gt;SERVER-77513&lt;/del&gt;&lt;/a&gt; Range deletion document removal commit must trigger in-memory state update&quot;&lt;/p&gt;

&lt;p&gt;This reverts commit 866f81bb3016d4db13771a2be827f17cb6520ea1.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/482bf3ab3660f0124a8bf5fba7d3fbeaba8c51a6&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/482bf3ab3660f0124a8bf5fba7d3fbeaba8c51a6&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5536594" author="xgen-internal-githook" created="Fri, 30 Jun 2023 13:02:52 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;Pierlauro Sciarelli&apos;, &apos;email&apos;: &apos;pierlauro.sciarelli@mongodb.com&apos;, &apos;username&apos;: &apos;pierlauro&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-77513&quot; title=&quot;Range deletion document removal commit must trigger in-memory state update&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-77513&quot;&gt;&lt;del&gt;SERVER-77513&lt;/del&gt;&lt;/a&gt; Range deletion document removal commit must trigger in-memory state update&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/866f81bb3016d4db13771a2be827f17cb6520ea1&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/866f81bb3016d4db13771a2be827f17cb6520ea1&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5454878" author="max.hirschhorn@10gen.com" created="Fri, 26 May 2023 13:35:46 +0000"  >&lt;blockquote&gt;
&lt;p&gt;The onDelete observer runs, removing the range deletion from in memory (which is a noop because it has not yet been added)&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;The storage transaction could always roll back. The ticket description implies RangeDeleterServiceOpObserver::onDelete() is implemented as updating the in&amp;#45;memory state speculatively. (And it doesn&apos;t register an onRollback() handler to account for this.) I feel like the correct fix would be to have RangeDeleterServiceOpObserver::onDelete() register an onCommit() handler to update the in&amp;#45;memory state. The pattern we have typically follow in the codebase which empirically is less error prone is to &quot;update on&amp;#45;disk state first, then update in&amp;#45;memory state second&quot;.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10520">
                    <name>Problem/Incident</name>
                                            <outwardlinks description="causes">
                                        <issuelink>
            <issuekey id="2505412">SERVER-83454</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is caused by">
                                        <issuelink>
            <issuekey id="2084613">SERVER-67849</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25133"><![CDATA[Sharding EMEA]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25578"><![CDATA[v7.0]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 26 May 2023 13:35:46 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        30 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>allison.easton@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            30 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>166.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>allison.easton@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>
            <customfieldvalue>pierlauro.sciarelli@mongodb.com</customfieldvalue>
            <customfieldvalue>Xgen-BuildBaron-User</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i2b0zj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i1t9uw:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="7188">Sharding EMEA 2023-06-12</customfieldvalue>
    <customfieldvalue id="7189">Sharding EMEA 2023-06-26</customfieldvalue>
    <customfieldvalue id="7190">Sharding EMEA 2023-07-10</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i2an4v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>