<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:26:20 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-74038] [Windows] Possible negative performance effects of SetProcessWorkingSetSize in SecureAllocator</title>
                <link>https://jira.mongodb.org/browse/SERVER-74038</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Our SecureAllocator attempts to lock memory pages so that the OS does not page them to disk. In Linux we use &lt;a href=&quot;https://github.com/mongodb/mongo/blob/8f36c90d997708507e1d73c4200d79a92619f422/src/mongo/base/secure_allocator.cpp#L253&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;mlock&lt;/a&gt;, and in Windows we use &lt;a href=&quot;https://github.com/mongodb/mongo/blob/8f36c90d997708507e1d73c4200d79a92619f422/src/mongo/base/secure_allocator.cpp#L181&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;VirtualLock&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Windows imposes limits on the number of pages that can be locked by VirtualLock. From the &lt;a href=&quot;https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-virtuallock?redirectedfrom=MSDN&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;documentation&lt;/a&gt;:&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;Each version of Windows has a limit on the maximum number of pages a process can lock. This limit is intentionally small to avoid severe performance degradation. Applications that need to lock larger numbers of pages must first call the SetProcessWorkingSetSize function to increase their minimum and maximum working set sizes.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;In &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-23705&quot; title=&quot;Number of databases on Windows is limited when using on-disk encryption&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-23705&quot;&gt;&lt;del&gt;SERVER-23705&lt;/del&gt;&lt;/a&gt;, we started using the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/8f36c90d997708507e1d73c4200d79a92619f422/src/mongo/base/secure_allocator.cpp#L153&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;SetProcessWorkingSetSize&lt;/a&gt; API when we reach these limits. &lt;/p&gt;

&lt;p&gt;Even when the pagefile is disabled, we have observed strange performance behavior where Windows moves essentially all of the mongod&apos;s memory from the Active state to the Inactive state. This is accompanied by very long stalls. In FTDC, the observed effect is that there is no mongod resident memory.&lt;/p&gt;

&lt;p&gt;My theory is that SetProcessWorkingSetSize tells the OS that certain memory is important, but by implication that everything outside of the working set is not actually important. As a result of setting the working set to a small value in order to lock pages, the also OS decides to mark the remaining resident memory as &quot;Inactive&quot; even if there is plenty of free memory available in the system. This has serious performance implications that we should investigate and understand.&lt;/p&gt;

&lt;p&gt;It seems like we either need to disprove this theory, find a different API, or use always use SetProcessWorkingSetSize to ensure that the entire process&apos;s memory is important.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2265387">SERVER-74038</key>
            <summary>[Windows] Possible negative performance effects of SetProcessWorkingSetSize in SecureAllocator</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="mark.benvenuto@mongodb.com">Mark Benvenuto</assignee>
                                    <reporter username="louis.williams@mongodb.com">Louis Williams</reporter>
                        <labels>
                    </labels>
                <created>Wed, 15 Feb 2023 14:23:16 +0000</created>
                <updated>Sun, 29 Oct 2023 21:26:16 +0000</updated>
                            <resolved>Mon, 27 Feb 2023 18:22:18 +0000</resolved>
                                                    <fixVersion>7.0.0-rc0</fixVersion>
                    <fixVersion>4.4.20</fixVersion>
                    <fixVersion>5.0.16</fixVersion>
                    <fixVersion>6.0.6</fixVersion>
                                                        <votes>0</votes>
                                    <watches>9</watches>
                                                                                                                <comments>
                            <comment id="5285790" author="xgen-internal-githook" created="Mon, 20 Mar 2023 17:58:59 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;Mark Benvenuto&apos;, &apos;email&apos;: &apos;mark.benvenuto@mongodb.com&apos;, &apos;username&apos;: &apos;markbenvenuto&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-74038&quot; title=&quot;[Windows] Possible negative performance effects of SetProcessWorkingSetSize in SecureAllocator&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-74038&quot;&gt;&lt;del&gt;SERVER-74038&lt;/del&gt;&lt;/a&gt; Grow dwMaximumWorkingSetSize with current working set size&lt;/p&gt;

&lt;p&gt;(cherry picked from commit db5ca2947f37d6706c01fe24d6294af75b6418c9)&lt;br/&gt;
Branch: v6.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/34de12fdfd56a07d1c1d6a1c194de6e6e906d1b0&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/34de12fdfd56a07d1c1d6a1c194de6e6e906d1b0&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5285733" author="xgen-internal-githook" created="Mon, 20 Mar 2023 17:44:30 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;Mark Benvenuto&apos;, &apos;email&apos;: &apos;mark.benvenuto@mongodb.com&apos;, &apos;username&apos;: &apos;markbenvenuto&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-74038&quot; title=&quot;[Windows] Possible negative performance effects of SetProcessWorkingSetSize in SecureAllocator&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-74038&quot;&gt;&lt;del&gt;SERVER-74038&lt;/del&gt;&lt;/a&gt; Grow dwMaximumWorkingSetSize with current working set size&lt;/p&gt;

&lt;p&gt;(cherry picked from commit db5ca2947f37d6706c01fe24d6294af75b6418c9)&lt;br/&gt;
Branch: v5.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/e41dfbe8bf58bc8489defe42dfb72a0512f5c2fe&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/e41dfbe8bf58bc8489defe42dfb72a0512f5c2fe&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5285678" author="xgen-internal-githook" created="Mon, 20 Mar 2023 17:30:04 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;Mark Benvenuto&apos;, &apos;email&apos;: &apos;mark.benvenuto@mongodb.com&apos;, &apos;username&apos;: &apos;markbenvenuto&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-74038&quot; title=&quot;[Windows] Possible negative performance effects of SetProcessWorkingSetSize in SecureAllocator&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-74038&quot;&gt;&lt;del&gt;SERVER-74038&lt;/del&gt;&lt;/a&gt; Grow dwMaximumWorkingSetSize with current working set size&lt;/p&gt;

&lt;p&gt;(cherry picked from commit db5ca2947f37d6706c01fe24d6294af75b6418c9)&lt;br/&gt;
Branch: v4.4&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/34023f77e285f99dfd92388a05c142855d6bbd4c&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/34023f77e285f99dfd92388a05c142855d6bbd4c&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5233766" author="xgen-internal-githook" created="Mon, 27 Feb 2023 18:12:03 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;Mark Benvenuto&apos;, &apos;email&apos;: &apos;mark.benvenuto@mongodb.com&apos;, &apos;username&apos;: &apos;markbenvenuto&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-74038&quot; title=&quot;[Windows] Possible negative performance effects of SetProcessWorkingSetSize in SecureAllocator&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-74038&quot;&gt;&lt;del&gt;SERVER-74038&lt;/del&gt;&lt;/a&gt; Grow dwMaximumWorkingSetSize with current working set size&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/db5ca2947f37d6706c01fe24d6294af75b6418c9&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/db5ca2947f37d6706c01fe24d6294af75b6418c9&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5222723" author="mark.benvenuto" created="Wed, 22 Feb 2023 20:58:04 +0000"  >&lt;p&gt;So the issue is that fundamentally that a pair of calls &lt;tt&gt;GetProcessWorkingSetSize/SetProcessWorkingSetSize&lt;/tt&gt; does not do what originally expected. The original fix (&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-23705&quot; title=&quot;Number of databases on Windows is limited when using on-disk encryption&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-23705&quot;&gt;&lt;del&gt;SERVER-23705&lt;/del&gt;&lt;/a&gt;) addressed the problem of exhausting the process quota for locked pages but did not appreciate the side effects of &lt;tt&gt;dwMaximumWorkingSetSize&lt;/tt&gt; in &lt;tt&gt;SetProcessWorkingSetSize&lt;/tt&gt;. Windows memory manager treats &lt;tt&gt;dwMaximumWorkingSetSize&lt;/tt&gt; as a target working set size for a process to achieve. This means that Windows tries to forcibly eject pages until it achieves its target size. The &lt;tt&gt;dwMaximumWorkingSetSize&lt;/tt&gt; that is returned by &lt;tt&gt;GetProcessWorkingSetSize&lt;/tt&gt; is not the current&#160;working set size of the process but the original process max working set size if the system is under memory pressure, default of 345 pages.&lt;/p&gt;

&lt;p&gt;This issue can be easily demonstrated by a custom repro using a unit test (See &lt;a href=&quot;https://github.com/mongodb/mongo/compare/master...markbenvenuto:mongo:secure_allocator_measure?expand=1&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/compare/master...markbenvenuto:mongo:secure_allocator_measure?expand=1&lt;/a&gt;). Said repro simply tries to gobble up memory with 1% of it being &quot;secure&quot; memory. By observing memory consumption from &lt;tt&gt;GetProcessWorkingSetSize&lt;/tt&gt; and &lt;tt&gt;GetProcessMemoryInfo&lt;/tt&gt;, we can see how the working set size oscillates as our test program consumes more memory and Windows is periodically told to empty the working set.&lt;/p&gt;

&lt;p&gt;&lt;ins&gt;Current Behavior&lt;/ins&gt;&lt;br/&gt;
&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;a id=&quot;433301_thumb&quot; href=&quot;https://jira.mongodb.org/secure/attachment/433301/433301_windows_bad_memory_trim.png&quot; title=&quot;windows_bad_memory_trim.png&quot; file-preview-type=&quot;image&quot; file-preview-id=&quot;433301&quot; file-preview-title=&quot;windows_bad_memory_trim.png&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/thumbnail/433301/_thumb_433301.png&quot; style=&quot;border: 0px solid black&quot; role=&quot;presentation&quot;/&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;&lt;ins&gt;Fixed Behavior&lt;/ins&gt;&lt;br/&gt;
 &lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;a id=&quot;433302_thumb&quot; href=&quot;https://jira.mongodb.org/secure/attachment/433302/433302_windows_good_memory_trim.png&quot; title=&quot;windows_good_memory_trim.png&quot; file-preview-type=&quot;image&quot; file-preview-id=&quot;433302&quot; file-preview-title=&quot;windows_good_memory_trim.png&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/thumbnail/433302/_thumb_433302.png&quot; style=&quot;border: 0px solid black&quot; role=&quot;presentation&quot;/&gt;&lt;/a&gt;&lt;/span&gt; &lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="279596">SERVER-23705</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="2272580">SERVER-74289</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="433301" name="windows_bad_memory_trim.png" size="116863" author="mark.benvenuto@mongodb.com" created="Wed, 22 Feb 2023 20:45:21 +0000"/>
                            <attachment id="433302" name="windows_good_memory_trim.png" size="68974" author="mark.benvenuto@mongodb.com" created="Wed, 22 Feb 2023 20:45:24 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25129"><![CDATA[Server Security]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23470"><![CDATA[v6.0]]></customfieldvalue>
    <customfieldvalue key="21777"><![CDATA[v5.0]]></customfieldvalue>
    <customfieldvalue key="18953"><![CDATA[v4.4]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 22 Feb 2023 20:58:04 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        46 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            46 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>
            <customfieldvalue>mark.benvenuto@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1w573:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i1ekpk:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="6920">Security 2023-03-06</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1vrcf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>