<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:59:25 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-22114] Mongos can accumulate multiple copies of ChunkManager when a shard restarts</title>
                <link>https://jira.mongodb.org/browse/SERVER-22114</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;When a shard restarts, it loses all its sharding metadata information. So if a mongos that has sharding information sends a write command to the restarted shard, it will get a stale config error from the shard because the shard contains no shard version (see note 1). The mongos will see the stale error with shard zero version from the response and will decide to perform a full reload (see note 2). In setups with very huge number of chunks (in millions), it takes time for the entire chunk metadata to be loaded and this can cause issues because this is done under a mutex, and will cause other threads which got the same response from the shard to decide to perform a full reload as well. This is exacerbated by the fact that the thread will have to acquire and release the same contentious mutex multiple times until it finishes, so it can hold on to the newly fully loaded data for even longer periods of time. Also note that for every full reload, it will create a new instance of ChunkManager and will try to atomically replace the old one with the new one when the reload finishes. In a mongos with multiple threads trying to execute a write command, it can create a situation where several threads will queue up trying to perform a full reload and some threads have loaded their own copy of the chunk metadata but are blocked waiting for the same mutex the other threads are waiting for the full reload. In certain cases with large enough chunks and simultaneous write command operations, it can spiral out of control, consume too much memory and ultimately get killed by the OOM killer in the operating system.&lt;/p&gt;

&lt;p&gt;Note 1:&lt;br/&gt;
In mongod write command execution path, version is checked first inside here:&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.2.0/src/mongo/db/commands/write_commands/batch_executor.cpp#L321&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/r3.2.0/src/mongo/db/commands/write_commands/batch_executor.cpp#L321&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;sets the error in the response, and then performs a refresh afterwards:&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.2.0/src/mongo/db/commands/write_commands/batch_executor.cpp#L361&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/r3.2.0/src/mongo/db/commands/write_commands/batch_executor.cpp#L361&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Note 2:&lt;br/&gt;
Shard response with zero version will result to unknown comparison result:&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.2.0/src/mongo/s/chunk_manager_targeter.cpp#L152&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/r3.2.0/src/mongo/s/chunk_manager_targeter.cpp#L152&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;and ultimately, causing it to flush the entire chunk manager:&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.2.0/src/mongo/s/chunk_manager_targeter.cpp#L674&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/r3.2.0/src/mongo/s/chunk_manager_targeter.cpp#L674&lt;/a&gt;&lt;/p&gt;</description>
                <environment></environment>
        <key id="246417">SERVER-22114</key>
            <summary>Mongos can accumulate multiple copies of ChunkManager when a shard restarts</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="randolph@mongodb.com">Randolph Tan</assignee>
                                    <reporter username="randolph@mongodb.com">Randolph Tan</reporter>
                        <labels>
                            <label>code-only</label>
                    </labels>
                <created>Sat, 9 Jan 2016 02:01:20 +0000</created>
                <updated>Wed, 25 Jan 2017 21:59:27 +0000</updated>
                            <resolved>Fri, 15 Jan 2016 22:09:55 +0000</resolved>
                                                    <fixVersion>3.0.9</fixVersion>
                    <fixVersion>3.2.3</fixVersion>
                    <fixVersion>3.3.1</fixVersion>
                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>12</watches>
                                                                                                                <comments>
                            <comment id="1145836" author="xgen-internal-githook" created="Sat, 16 Jan 2016 06:39:52 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;renctan&apos;, u&apos;name&apos;: u&apos;Randolph Tan&apos;, u&apos;email&apos;: u&apos;randolph@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-22114&quot; title=&quot;Mongos can accumulate multiple copies of ChunkManager when a shard restarts&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-22114&quot;&gt;&lt;del&gt;SERVER-22114&lt;/del&gt;&lt;/a&gt; fix code format&lt;br/&gt;
Branch: v3.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/4946e2dd15dc97d0c884d04f63a8f9470839786f&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/4946e2dd15dc97d0c884d04f63a8f9470839786f&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1145644" author="xgen-internal-githook" created="Fri, 15 Jan 2016 22:07:50 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;renctan&apos;, u&apos;name&apos;: u&apos;Randolph Tan&apos;, u&apos;email&apos;: u&apos;randolph@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-22114&quot; title=&quot;Mongos can accumulate multiple copies of ChunkManager when a shard restarts&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-22114&quot;&gt;&lt;del&gt;SERVER-22114&lt;/del&gt;&lt;/a&gt; Avoid doing unnecessary repeated calls to DBConfig::_load&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 0f85256eefa78b1b74b9e6ae0d1c646853f6f22f)&lt;br/&gt;
Branch: v3.2&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/d74c7ae40f82bd5bf6334c8114c0cf4948f60002&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/d74c7ae40f82bd5bf6334c8114c0cf4948f60002&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1145643" author="xgen-internal-githook" created="Fri, 15 Jan 2016 22:07:49 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;renctan&apos;, u&apos;name&apos;: u&apos;Randolph Tan&apos;, u&apos;email&apos;: u&apos;randolph@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-22114&quot; title=&quot;Mongos can accumulate multiple copies of ChunkManager when a shard restarts&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-22114&quot;&gt;&lt;del&gt;SERVER-22114&lt;/del&gt;&lt;/a&gt; Remove unnecessary refresh in ChunkManagerTargeter::refreshNow&lt;/p&gt;

&lt;p&gt;In addition, limit the lifetime of ChunkManager instances so it can free references to ChunkManager earlier.&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 65f2da2c49c2b22d2b80e6562b9b61f242cb9a18)&lt;br/&gt;
Branch: v3.2&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/927caf6e06d77adda6caa23120fc08b20059151b&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/927caf6e06d77adda6caa23120fc08b20059151b&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1145608" author="xgen-internal-githook" created="Fri, 15 Jan 2016 21:45:09 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;renctan&apos;, u&apos;name&apos;: u&apos;Randolph Tan&apos;, u&apos;email&apos;: u&apos;randolph@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-22114&quot; title=&quot;Mongos can accumulate multiple copies of ChunkManager when a shard restarts&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-22114&quot;&gt;&lt;del&gt;SERVER-22114&lt;/del&gt;&lt;/a&gt; Avoid doing unnecessary repeated calls to DBConfig::_load&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 0f85256eefa78b1b74b9e6ae0d1c646853f6f22f)&lt;/p&gt;

&lt;p&gt;Conflicts:&lt;br/&gt;
	src/mongo/s/config.cpp&lt;br/&gt;
	src/mongo/s/config.h&lt;br/&gt;
Branch: v3.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/aaa56a5d37ee1ad78326a2e22e097bdf91bedeed&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/aaa56a5d37ee1ad78326a2e22e097bdf91bedeed&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1145607" author="xgen-internal-githook" created="Fri, 15 Jan 2016 21:45:08 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;renctan&apos;, u&apos;name&apos;: u&apos;Randolph Tan&apos;, u&apos;email&apos;: u&apos;randolph@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-22114&quot; title=&quot;Mongos can accumulate multiple copies of ChunkManager when a shard restarts&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-22114&quot;&gt;&lt;del&gt;SERVER-22114&lt;/del&gt;&lt;/a&gt; Remove unnecessary refresh in ChunkManagerTargeter::refreshNow&lt;/p&gt;

&lt;p&gt;In addition, limit the lifetime of ChunkManager instances so it can free references to ChunkManager earlier.&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 65f2da2c49c2b22d2b80e6562b9b61f242cb9a18)&lt;/p&gt;

&lt;p&gt;Conflicts:&lt;br/&gt;
	src/mongo/s/chunk_manager_targeter.cpp&lt;br/&gt;
	src/mongo/s/chunk_manager_targeter.h&lt;br/&gt;
	src/mongo/s/cluster_write.cpp&lt;br/&gt;
	src/mongo/s/cluster_write.h&lt;br/&gt;
	src/mongo/s/commands/cluster_write_cmd.cpp&lt;br/&gt;
	src/mongo/s/write_ops/batch_write_exec.cpp&lt;br/&gt;
	src/mongo/s/write_ops/batch_write_exec.h&lt;br/&gt;
	src/mongo/s/write_ops/batch_write_exec_test.cpp&lt;br/&gt;
Branch: v3.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/56eae69cf882d92e502340c9ce5dcc4a7058db9a&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/56eae69cf882d92e502340c9ce5dcc4a7058db9a&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1145374" author="xgen-internal-githook" created="Fri, 15 Jan 2016 18:36:14 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;renctan&apos;, u&apos;name&apos;: u&apos;Randolph Tan&apos;, u&apos;email&apos;: u&apos;randolph@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-22114&quot; title=&quot;Mongos can accumulate multiple copies of ChunkManager when a shard restarts&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-22114&quot;&gt;&lt;del&gt;SERVER-22114&lt;/del&gt;&lt;/a&gt; Fix comment&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/7407700466ea8f2f281c3daad50e7a2b42d4f460&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/7407700466ea8f2f281c3daad50e7a2b42d4f460&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1145242" author="xgen-internal-githook" created="Fri, 15 Jan 2016 17:07:34 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;renctan&apos;, u&apos;name&apos;: u&apos;Randolph Tan&apos;, u&apos;email&apos;: u&apos;randolph@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-22114&quot; title=&quot;Mongos can accumulate multiple copies of ChunkManager when a shard restarts&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-22114&quot;&gt;&lt;del&gt;SERVER-22114&lt;/del&gt;&lt;/a&gt; Avoid doing unnecessary repeated calls to DBConfig::_load&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/0f85256eefa78b1b74b9e6ae0d1c646853f6f22f&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/0f85256eefa78b1b74b9e6ae0d1c646853f6f22f&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1145241" author="xgen-internal-githook" created="Fri, 15 Jan 2016 17:07:33 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;renctan&apos;, u&apos;name&apos;: u&apos;Randolph Tan&apos;, u&apos;email&apos;: u&apos;randolph@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-22114&quot; title=&quot;Mongos can accumulate multiple copies of ChunkManager when a shard restarts&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-22114&quot;&gt;&lt;del&gt;SERVER-22114&lt;/del&gt;&lt;/a&gt; Remove unnecessary refresh in ChunkManagerTargeter::refreshNow&lt;/p&gt;

&lt;p&gt;In addition, limit the lifetime of ChunkManager instances so it can free references to ChunkManager earlier.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/65f2da2c49c2b22d2b80e6562b9b61f242cb9a18&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/65f2da2c49c2b22d2b80e6562b9b61f242cb9a18&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="282924">SERVER-23958</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="283060">SERVER-23965</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12451" key="com.atlassian.jira.plugin.system.customfieldtypes:multiversion">
                        <customfieldname>Backport Completed</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="16007">3.0.9</customfieldvalue>
    <customfieldvalue id="15909">3.2.3</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 15 Jan 2016 17:07:33 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        8 years, 4 weeks, 4 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_14262" key="com.atlassian.jira.plugin.system.customfieldtypes:datepicker">
                        <customfieldname>End date</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 15 Jan 2016 23:59:59 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>crystal.horn@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            8 years, 4 weeks, 4 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>randolph@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrkl07:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hsfbrr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="841">Sharding F (01/29/16)</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_14261" key="com.atlassian.jira.plugin.system.customfieldtypes:datepicker">
                        <customfieldname>Start date</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 11 Jan 2016 00:00:00 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsfbwf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>