<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:04:39 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-3996] Backport fix for SERVER-3002 to v1.8 branch</title>
                <link>https://jira.mongodb.org/browse/SERVER-3996</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We seem to be encountering this issue in v1.8.2 and wanted to see about backporting the fix for &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3002&quot; title=&quot;mongos memory error CursorCache iterator error&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3002&quot;&gt;&lt;del&gt;SERVER-3002&lt;/del&gt;&lt;/a&gt; to the v1.8 branch.  Looking at the v1.8 branch, I do not see a similar fix for calling the increment operator on an erased iterator: &lt;a href=&quot;https://github.com/mongodb/mongo/blob/v1.8/s/cursors.cpp#L268&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/v1.8/s/cursors.cpp#L268&lt;/a&gt;.  &lt;/p&gt;

&lt;p&gt;The fix in the v2 (related to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3002&quot; title=&quot;mongos memory error CursorCache iterator error&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3002&quot;&gt;&lt;del&gt;SERVER-3002&lt;/del&gt;&lt;/a&gt;) starts iterating again from the beginning: &lt;a href=&quot;https://github.com/mongodb/mongo/blob/v2.0/s/cursors.cpp#L277&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/v2.0/s/cursors.cpp#L277&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;While that fix is perfectly valid and seems like it could easily be backported, I was also curious a c++11 feature like map::erase()&apos;s return iterator is allowed in the current Mongo source.  My thinking is we can replace the fix here:&lt;/p&gt;

&lt;p&gt;_cursors.erase( i );&lt;br/&gt;
i = _cursors.begin(); // possible 2nd entry will get skipped, will get on next pass&lt;/p&gt;

&lt;p&gt;with:&lt;/p&gt;

&lt;p&gt;i = _cursors.erase&lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/information.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;;&lt;/p&gt;

&lt;p&gt;Though perhaps I&apos;m missing another reason where it&apos;s better to start from the begin()nig...  I assumed the scoped lock for _mutex would prevent any updates to the _cursors map, and it looks like the timeout check uses the &apos;now&apos; value acquired before we start iterating.  Tangentially, it looks like CursorCache::~CursorCache() doesn&apos;t acquire the _mutex lock before checking _cursors.size(), though I&apos;m unsure if this is actually an issue.&lt;/p&gt;

&lt;p&gt;Anyway, the issue we&apos;re seeing just started recently, and has been observed on 5 out of 6 servers that run mongos (all nearly identical ec2 instances).  Here&apos;s the portion of the log file that contains the error, and apologies in advance for the lack of verbose output (occurs only on our production servers, but I can enable verbose logging if it will help).&lt;/p&gt;


&lt;p&gt;Sat Oct  1 05:00:16 &lt;span class=&quot;error&quot;&gt;&amp;#91;cursorTimeout&amp;#93;&lt;/span&gt; killing old cursor 3588663744748048245 idle for: 600028ms&lt;br/&gt;
Received signal 11&lt;br/&gt;
Backtrace: 0x52f8f5 0x7f65d137aaf0 0x7f65d1bd6533 0x6711c5 0x526a03 0x50454b 0x505e04 0x6a50a0 0x7f65d1e7e9ca 0x7f65d142d70d&lt;br/&gt;
/usr/bin/mongos(_ZN5mongo17printStackAndExitEi+0x75)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x52f8f5&amp;#93;&lt;/span&gt;&lt;br/&gt;
/lib/libc.so.6(+0x33af0)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x7f65d137aaf0&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/lib/libstdc++.so.6(_ZSt18_Rb_tree_incrementPSt18_Rb_tree_node_base+0x13)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x7f65d1bd6533&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(_ZN5mongo11CursorCache10doTimeoutsEv+0x75)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x6711c5&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(_ZN5mongo4task4Task3runEv+0x33)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x526a03&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(_ZN5mongo13BackgroundJob7jobBodyEN5boost10shared_ptrINS0_9JobStatusEEE+0x12b)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x50454b&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(_ZN5boost6detail11thread_dataINS_3_bi6bind_tIvNS_4_mfi3mf1IvN5mongo13BackgroundJobENS_10shared_ptrINS7_9JobStatusEEEEENS2_5list2INS2_5valueIPS7_EENSD_ISA_EEEEEEE3runEv+0x7&lt;br/&gt;
4)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x505e04&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(thread_proxy+0x80)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x6a50a0&amp;#93;&lt;/span&gt;&lt;br/&gt;
/lib/libpthread.so.0(+0x69ca)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x7f65d1e7e9ca&amp;#93;&lt;/span&gt;&lt;br/&gt;
/lib/libc.so.6(clone+0x6d)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x7f65d142d70d&amp;#93;&lt;/span&gt;&lt;br/&gt;
===&lt;br/&gt;
Received signal 11&lt;br/&gt;
Backtrace: 0x52f8f5 0x7f65d137aaf0 0x532ea0 0x577654 0x577c71 0x630a9e 0x6361c8 0x66841c 0x67d187 0x580b7c 0x6a50a0 0x7f65d1e7e9ca 0x7f65d142d70d &lt;br/&gt;
/usr/bin/mongos(_ZN5mongo17printStackAndExitEi+0x75)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x52f8f5&amp;#93;&lt;/span&gt;&lt;br/&gt;
/lib/libc.so.6(+0x33af0)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x7f65d137aaf0&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(_ZN5mongo16DBConnectionPool11onHandedOutEPNS_12DBClientBaseE+0x20)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x532ea0&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(_ZN5mongo15ShardConnection5_initEv+0x1b4)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x577654&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(_ZN5mongo15ShardConnectionC1ERKNS_5ShardERKSs+0xa1)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x577c71&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(_ZN5mongo8Strategy7doQueryERNS_7RequestERKNS_5ShardE+0x4e)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x630a9e&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(_ZN5mongo14SingleStrategy7queryOpERNS_7RequestE+0x4d8)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x6361c8&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(_ZN5mongo7Request7processEi+0x29c)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x66841c&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x77)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x67d187&amp;#93;&lt;/span&gt;&lt;br/&gt;
Sat Oct  1 05:00:16  CursorCache at shutdown -  sharded: 543 passthrough: 0&lt;/p&gt;


&lt;p&gt;If this segv appears unrelated to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3002&quot; title=&quot;mongos memory error CursorCache iterator error&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3002&quot;&gt;&lt;del&gt;SERVER-3002&lt;/del&gt;&lt;/a&gt;, please let me know and I&apos;ll report more information.  Thank you for your time!&lt;/p&gt;
</description>
                <environment>Ubuntu 10.04&lt;br/&gt;
EC2 large instance&lt;br/&gt;
mongos v1.8.2 (from 10gen apt source)&lt;br/&gt;
php driver v1.0.11&lt;br/&gt;
mongos is connected to 2 shards along with 7 other instances of mongos on other hosts.</environment>
        <key id="23117">SERVER-3996</key>
            <summary>Backport fix for SERVER-3002 to v1.8 branch</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="greg.mckeon@mongodb.com">Gregory McKeon</assignee>
                                    <reporter username="benjamin.becker">Ben Becker</reporter>
                        <labels>
                            <label>CursorCache</label>
                            <label>cursor</label>
                            <label>timeout</label>
                    </labels>
                <created>Sat, 1 Oct 2011 19:29:32 +0000</created>
                <updated>Thu, 2 Aug 2018 21:26:12 +0000</updated>
                            <resolved>Thu, 20 Oct 2011 19:57:58 +0000</resolved>
                                    <version>1.8.2</version>
                                    <fixVersion>1.8.4</fixVersion>
                                    <component>Sharding</component>
                                        <votes>3</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="61688" author="greg_10gen" created="Thu, 20 Oct 2011 19:57:04 +0000"  >&lt;p&gt;backported to 1.8.4&lt;/p&gt;</comment>
                            <comment id="61686" author="auto" created="Thu, 20 Oct 2011 19:56:13 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;login&apos;: u&apos;gregstuder&apos;, u&apos;name&apos;: u&apos;gregs&apos;, u&apos;email&apos;: u&apos;greg@10gen.com&apos;}
&lt;p&gt;Message: backport of fix for cursor timeout iteration, from jira discussion &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3996&quot; title=&quot;Backport fix for SERVER-3002 to v1.8 branch&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3996&quot;&gt;&lt;del&gt;SERVER-3996&lt;/del&gt;&lt;/a&gt;&lt;br/&gt;
Branch: v1.8&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/85902e0e826ba543f9ea6021e0d370fcc6dd6854&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/85902e0e826ba543f9ea6021e0d370fcc6dd6854&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="61022" author="benjamin.becker" created="Mon, 17 Oct 2011 23:22:58 +0000"  >&lt;p&gt;Carlos,&lt;br/&gt;
Interesting, I have not seen those two errors you got after applying the patch...  For me, mongoS was running for months before some instances started hitting this segfault (avg of 1 in 7 servers segv each day).  I no longer get the segfault, but all other log output appears the same as it did before the patch, including some odd cursor related messages.  So I&apos;m starting to speculate this bug was unmasked as a symptom of the larger problem.&lt;/p&gt;

&lt;p&gt;I&apos;m afraid I don&apos;t have much insight to offer at this point, but if time permits I&apos;ll dig further into the cursor log entries and report back (or open a new issue).&lt;/p&gt;</comment>
                            <comment id="60928" author="carlos8f" created="Mon, 17 Oct 2011 20:00:37 +0000"  >&lt;p&gt;Ben, I tried applying the patch in your fork to 1.8 nightly, but after switching to the patched mongos, a few hours later saw a slew of &quot;too many attempts to update config&quot; exceptions from the php driver, which is at 1.2.6 (could be &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3739&quot; title=&quot;mongos: &amp;quot;too many attempts to update config, failing&amp;quot;&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3739&quot;&gt;&lt;del&gt;SERVER-3739&lt;/del&gt;&lt;/a&gt; ?). I then tried patching 1.8.3, but upon deploying got flooded in &quot;can&apos;t send query&quot; exceptions from the driver. Have not seen either of those errors with the standard 1.8.3 mongos. Still getting &quot;killing old cursor&quot; appearing in the mongos log about 10 times a second, with or without the patch. I&apos;m really not sure what to do, other than try upgrading the cluster to 2.0 again (my first attempt had to be reverted due to poor query performance). Seems like everytime I get one bug fixed, 2 more pop up :-/&lt;/p&gt;</comment>
                            <comment id="60924" author="benjamin.becker" created="Mon, 17 Oct 2011 19:48:53 +0000"  >&lt;p&gt;Just to follow up:  All servers have been up and running with the supplied v1.8.2 patch for over a week without incident.  The optimization for the v2 branch has not been tested yet.&lt;/p&gt;</comment>
                            <comment id="59236" author="benjamin.becker" created="Fri, 7 Oct 2011 23:24:29 +0000"  >&lt;p&gt;Submitted github pull requests for the patch to be applied to the v1.8 branch.&lt;/p&gt;

&lt;p&gt;Submitted a secondary pull request to replace the original patch for &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3002&quot; title=&quot;mongos memory error CursorCache iterator error&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3002&quot;&gt;&lt;del&gt;SERVER-3002&lt;/del&gt;&lt;/a&gt; with a fix that doesn&apos;t require restarting from the beginning on erase().&lt;/p&gt;

&lt;p&gt;All testing has been successful with the supplied patch so far, but no testing has been done with the same patch against v2.&lt;/p&gt;</comment>
                            <comment id="58877" author="benjamin.becker" created="Thu, 6 Oct 2011 19:24:38 +0000"  >&lt;p&gt;Just to update:  I&apos;ve pushed this to two more servers, and the only server that has crashed with the segv in CursorCache::doTimeout is the unpatched one.&lt;/p&gt;</comment>
                            <comment id="58494" author="benjamin.becker" created="Tue, 4 Oct 2011 23:26:22 +0000"  >&lt;p&gt;The supplied patch has held up for 3 days now with no trouble so far.  It&apos;s been running alongside other servers without the patch, and thus far 3 out of 5 non-patched mongos instances have crashed in CursorCache::doTimeouts, and the patched server has been running fine (no noticeable mem leaks, db connections seem stable, etc).&lt;/p&gt;

&lt;p&gt;BTW, it would be great to sneak in this patch (or a backport of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3002&quot; title=&quot;mongos memory error CursorCache iterator error&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3002&quot;&gt;&lt;del&gt;SERVER-3002&lt;/del&gt;&lt;/a&gt;) into 1.8.4! &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/smile.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/p&gt;

&lt;p&gt;     Best,&lt;br/&gt;
     Ben&lt;/p&gt;</comment>
                            <comment id="58491" author="carlos8f" created="Tue, 4 Oct 2011 23:14:58 +0000"  >&lt;p&gt;I think I&apos;m running into the same bug, which periodically crashes our mongos servers when under high activity.&lt;/p&gt;

&lt;p&gt;Mon Oct  3 07:33:40 &lt;span class=&quot;error&quot;&gt;&amp;#91;cursorTimeout&amp;#93;&lt;/span&gt; killing old cursor 5371030602413797610 idle for: 600102ms&lt;br/&gt;
Mon Oct  3 07:33:40 &lt;span class=&quot;error&quot;&gt;&amp;#91;cursorTimeout&amp;#93;&lt;/span&gt; killing old cursor 5935414358782317860 idle for: 600322ms&lt;br/&gt;
Received signal 11&lt;br/&gt;
Backtrace: 0x52f595 0x7f974b81ec20 0x7f974c0739c3 0x673de5 0x5266a3 0x5041eb 0x505aa4 0x6a7f70 0x7f974c314971 0x7f974b8d192d &lt;br/&gt;
/usr/bin/mongos(_ZN5mongo17printStackAndExitEi+0x75)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x52f595&amp;#93;&lt;/span&gt;&lt;br/&gt;
/lib/libc.so.6(+0x33c20)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x7f974b81ec20&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/lib/libstdc++.so.6(_ZSt18_Rb_tree_incrementPSt18_Rb_tree_node_base+0x13)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x7f974c0739c3&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(_ZN5mongo11CursorCache10doTimeoutsEv+0x75)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x673de5&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(_ZN5mongo4task4Task3runEv+0x33)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x5266a3&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(_ZN5mongo13BackgroundJob7jobBodyEN5boost10shared_ptrINS0_9JobStatusEEE+0x12b)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x5041eb&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(_ZN5boost6detail11thread_dataINS_3_bi6bind_tIvNS_4_mfi3mf1IvN5mongo13BackgroundJobENS_10shared_ptrINS7_9JobStatusEEEEENS2_5list2INS2_5valueIPS7_EENSD_ISA_EEEEEEE3runEv+0x74)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x505aa4&amp;#93;&lt;/span&gt;&lt;br/&gt;
/usr/bin/mongos(thread_proxy+0x80)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x6a7f70&amp;#93;&lt;/span&gt;&lt;br/&gt;
/lib/libpthread.so.0(+0x7971)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x7f974c314971&amp;#93;&lt;/span&gt;&lt;br/&gt;
/lib/libc.so.6(clone+0x6d)&lt;span class=&quot;error&quot;&gt;&amp;#91;0x7f974b8d192d&amp;#93;&lt;/span&gt;&lt;br/&gt;
===&lt;br/&gt;
Mon Oct  3 07:33:41  CursorCache at shutdown -  sharded: 196 passthrough: 7&lt;/p&gt;

&lt;p&gt;Upgrading to 2.0 is not an option for us at this point, we tried it and there were disastrous performance problems. Would love to get this in the 1.8 branch.&lt;/p&gt;

&lt;p&gt;@Ben, let me know how the patch is faring so far. I may have to use it if 1.8.4 doesn&apos;t land soon. Thanks &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/smile.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/p&gt;</comment>
                            <comment id="58103" author="benjamin.becker" created="Sun, 2 Oct 2011 08:26:41 +0000"  >&lt;p&gt;Please disregard the c++11 comment earlier; easier to change the for-loop to a while-loop and post increment the iterator in the call to erase().  Currently testing a custom build from the 1.8.2 tag with the attached patch applied.  I will leave this running alongside vanilla v1.8.2 servers to verify this fixes the issue reported.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="13378" name="mongo_doTimeouts_fix.diff" size="844" author="benjamin.becker" created="Sun, 2 Oct 2011 08:33:30 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>10.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 4 Oct 2011 23:14:58 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        12 years, 17 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>greg.mckeon@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            12 years, 17 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>auto</customfieldvalue>
            <customfieldvalue>benjamin.becker</customfieldvalue>
            <customfieldvalue>carlos8f</customfieldvalue>
            <customfieldvalue>greg.mckeon@mongodb.com</customfieldvalue>
            <customfieldvalue>greg_10gen</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrop0f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrm37z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>42431</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht0drr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>