<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:21:41 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-9868] heartbeats not responded to during mmap flushing on Windows</title>
                <link>https://jira.mongodb.org/browse/SERVER-9868</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We have a environment with 2 nodes and 1 arbiter with the following configuration:&lt;/p&gt;

&lt;p&gt;{&lt;br/&gt;
        &quot;_id&quot; : &quot;rssc01&quot;,&lt;br/&gt;
        &quot;version&quot; : 3,&lt;br/&gt;
        &quot;members&quot; : [&lt;br/&gt;
                &lt;/p&gt;
{
                        &quot;_id&quot; : 0,
                        &quot;host&quot; : &quot;LOG-MNGSC11:27017&quot;
                }
&lt;p&gt;,&lt;br/&gt;
                &lt;/p&gt;
{
                        &quot;_id&quot; : 1,
                        &quot;host&quot; : &quot;log-mngsc21:27017&quot;
                }
&lt;p&gt;,&lt;/p&gt;
                {
                        &quot;_id&quot; : 2,
                        &quot;host&quot; : &quot;log-mngsc22:27018&quot;,
                        &quot;arbiterOnly&quot; : true
                }
&lt;p&gt;        ]&lt;br/&gt;
}&lt;/p&gt;

&lt;p&gt;LOG-MNGSC11 is the primary and LOG-MNGSC21 is the secondary.&lt;/p&gt;

&lt;p&gt;Suddenly, the replication fails with the following message on secondary:&lt;/p&gt;

&lt;p&gt;Thu Jun 06 11:56:50.656 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsHealthPoll&amp;#93;&lt;/span&gt; replset info LOG-MNGSC11:27017 thinks that we are down&lt;br/&gt;
Thu Jun 06 11:56:52.310 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsHealthPoll&amp;#93;&lt;/span&gt; replset info log-mngsc22:27018 thinks that we are down&lt;br/&gt;
Thu Jun 06 11:56:52.481 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn12160&amp;#93;&lt;/span&gt; command admin.$cmd command: &lt;/p&gt;
{ writebacklisten: ObjectId(&apos;51afe4406127103ae52a9fc0&apos;) }
&lt;p&gt; ntoreturn:1 keyUpdates:0  reslen:44 300005ms&lt;br/&gt;
Thu Jun 06 11:56:52.668 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsHealthPoll&amp;#93;&lt;/span&gt; replset info LOG-MNGSC11:27017 thinks that we are down&lt;br/&gt;
Thu Jun 06 11:56:52.793 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsBackgroundSync&amp;#93;&lt;/span&gt; Socket recv() timeout  172.29.106.92:27017&lt;br/&gt;
Thu Jun 06 11:56:52.793 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsBackgroundSync&amp;#93;&lt;/span&gt; SocketException: remote: 172.29.106.92:27017 error: 9001 socket exception &lt;span class=&quot;error&quot;&gt;&amp;#91;3&amp;#93;&lt;/span&gt; server &lt;span class=&quot;error&quot;&gt;&amp;#91;172.29.106.92:27017&amp;#93;&lt;/span&gt; &lt;br/&gt;
Thu Jun 06 11:56:52.793 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsBackgroundSync&amp;#93;&lt;/span&gt; DBClientCursor::init call() failed&lt;/p&gt;

&lt;p&gt;At primary, I see the following messages:&lt;/p&gt;

&lt;p&gt;Thu Jun 06 11:56:52.524 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; connection accepted from 172.29.106.95:56714 #16239 (66 connections now open)&lt;br/&gt;
Thu Jun 06 11:56:55.847 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsHealthPoll&amp;#93;&lt;/span&gt; DBClientCursor::init call() failed&lt;br/&gt;
Thu Jun 06 11:56:57.329 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn16236&amp;#93;&lt;/span&gt; query local.oplog.rs query: { ts: &lt;/p&gt;
{ $gte: Timestamp 1370511563000|895 }
&lt;p&gt; } cursorid:479480611067557781 ntoreturn:0 ntoskip:0 nscanned:102 keyUpdates:0 numYields: 2264 locks(micros) r:727945 nreturned:101 reslen:12039 35319ms&lt;br/&gt;
Thu Jun 06 11:56:57.329 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn16236&amp;#93;&lt;/span&gt; end connection 172.29.106.95:56704 (65 connections now open)&lt;/p&gt;</description>
                <environment>Thu Jun 06 13:21:10.788 [initandlisten] MongoDB starting : pid=7356 port=27017 dbpath=e:\mongodb_rssc01\data\db 64-bit host=LOG-MNGSC21&lt;br/&gt;
Thu Jun 06 13:21:10.788 [initandlisten] db version v2.4.3&lt;br/&gt;
Thu Jun 06 13:21:10.788 [initandlisten] git version: fe1743177a5ea03e91e0052fb5e2cb2945f6d95f&lt;br/&gt;
Thu Jun 06 13:21:10.788 [initandlisten] build info: windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack=&amp;#39;Service Pack 1&amp;#39;) BOOST_LIB_VERSION=1_49&lt;br/&gt;
</environment>
        <key id="78098">SERVER-9868</key>
            <summary>heartbeats not responded to during mmap flushing on Windows</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="mark.benvenuto@mongodb.com">Mark Benvenuto</assignee>
                                    <reporter username="david.verdejo@logitravel.com">David Verdejo</reporter>
                        <labels>
                            <label>DBClientCursor</label>
                            <label>rsHealthPoll</label>
                    </labels>
                <created>Thu, 6 Jun 2013 11:44:54 +0000</created>
                <updated>Tue, 16 Nov 2021 13:22:21 +0000</updated>
                            <resolved>Tue, 8 Jul 2014 15:54:34 +0000</resolved>
                                    <version>2.4.3</version>
                                                    <component>Replication</component>
                                        <votes>2</votes>
                                    <watches>10</watches>
                                                                                                                <comments>
                            <comment id="535924" author="thomasr" created="Fri, 4 Apr 2014 17:26:41 +0000"  >&lt;p&gt;I believe the original poster&apos;s replica set failover was due to very long flushes&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Fri Jun  6 11:48:34.274 [DataFileSync] flushing mmaps took 439956ms for 17 files&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;during which the heartbeats and other requests were not processed (Windows platform). The flushes here were caused by a multi remove (here log entry from primary). The time of the &quot;thinks that we are down&quot; messages correlates directly with the period of the removes.&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Fri Jun  6 13:21:19.677 [conn16132] remove cachehoteles.Cache_AGHSmart_XMLHotel ndeleted:3876379 keyUpdates:0 numYields: 17683 locks(micros) w:5953725444 6558335ms&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;This is similar to what we see on &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-12880&quot; title=&quot;Server pauses on requests every 60 seconds&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-12880&quot;&gt;&lt;del&gt;SERVER-12880&lt;/del&gt;&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-13444&quot; title=&quot;Long locked flush without inserts and updates&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-13444&quot;&gt;&lt;del&gt;SERVER-13444&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="445724" author="sflint@sailthru.com" created="Wed, 23 Oct 2013 22:38:25 +0000"  >&lt;p&gt;I am experiencing this issue in production and I have 6TB of data on this system, so re-syncing the data isn&apos;t as easy and just doing it.   I am running 2.4.4 and the repl lag is about 57 hours behind now.  Please advise asap.&lt;/p&gt;</comment>
                            <comment id="438441" author="sidd@tinyurl.com" created="Wed, 9 Oct 2013 20:26:31 +0000"  >&lt;p&gt;Hi yes we are actually running that version.  The problem seems quite random and in the current setup I have all reads going to the primary so it&apos;s not like there&apos;s some long running query that might cause this.  Is there something that I can do to figure out what&apos;s going on that causes the issue to happen?  When the issue does happen I always see these sequence of messages:&lt;/p&gt;

&lt;p&gt;Wed Oct  9 20:16:58.563 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsBackgroundSync&amp;#93;&lt;/span&gt; Socket recv() timeout  172.17.0.85:27017&lt;br/&gt;
Wed Oct  9 20:16:58.563 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsBackgroundSync&amp;#93;&lt;/span&gt; SocketException: remote: 172.17.0.85:27017 error: 9001 socket exception &lt;span class=&quot;error&quot;&gt;&amp;#91;RECV_TIMEOUT&amp;#93;&lt;/span&gt; server &lt;span class=&quot;error&quot;&gt;&amp;#91;172.17.0.85:27017&amp;#93;&lt;/span&gt; &lt;br/&gt;
Wed Oct  9 20:16:58.563 &lt;span class=&quot;error&quot;&gt;&amp;#91;rsBackgroundSync&amp;#93;&lt;/span&gt; DBClientCursor::init call() failed&lt;/p&gt;</comment>
                            <comment id="437888" author="dan@10gen.com" created="Wed, 9 Oct 2013 03:40:40 +0000"  >&lt;p&gt;The underlying bug referred to in this ticket was resolved in the latest version of MongoDB (v2.4.6).  Are you able to try running with this version?&lt;/p&gt;</comment>
                            <comment id="435509" author="sidd@tinyurl.com" created="Thu, 3 Oct 2013 22:45:43 +0000"  >&lt;p&gt;Hi,&lt;/p&gt;

&lt;p&gt;I&apos;ve run into the same exact problem multiple times in my setup which is also a setup with 2 replicas and 1 arbiter.  I run into this problem on the secondary quite often which requires an entire resync in order to fix the issue.  I&apos;m running on FreeBSD 9.2-RC4 on a ZFS filesystem if that helps.  Is there an equivalent gdb command to run on FreeBSD to gather thread dumps?&lt;/p&gt;</comment>
                            <comment id="389149" author="david.verdejo@logitravel.com" created="Fri, 26 Jul 2013 06:25:37 +0000"  >&lt;p&gt;Hello,&lt;/p&gt;

&lt;p&gt;Please, could you send me the command to execute on Windows?&lt;/p&gt;
</comment>
                            <comment id="388612" author="sam.helman@10gen.com" created="Thu, 25 Jul 2013 14:46:28 +0000"  >&lt;p&gt;Hello,&lt;/p&gt;

&lt;p&gt;Sorry for the response delay.  The issue that both of you are seeing looks like it may be &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-10085&quot; title=&quot;Heartbeats can time out due to high network latency while fetching oplog batches&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-10085&quot;&gt;&lt;del&gt;SERVER-10085&lt;/del&gt;&lt;/a&gt;, which causes heartbeat requests to time out if there is a long-running oplog request pending.  If you are able to gather some more information for us and post or attach it, we can take a look and see if it points to the issue we expect.&lt;/p&gt;

&lt;p&gt;The information we need can be obtained by running &lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;gdb --pid $MONGODB_PID --batch --eval-command=&apos;thread apply all bt&apos;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;(replacing $MONGODB_PID with the pid of the running process).  This will provide stack trace info from gdb on all the currently running threads.  Ideally, we would like the information from times that the process is healthy and reachable, as well as from times it is unreachable, so we can compare.  &lt;/p&gt;

&lt;p&gt;Thanks!&lt;/p&gt;</comment>
                            <comment id="384080" author="byteofthat" created="Thu, 18 Jul 2013 16:42:13 +0000"  >&lt;p&gt;I am also experiencing this exact same issue. Any resolution as of yet?&lt;/p&gt;</comment>
                            <comment id="354388" author="david.verdejo@logitravel.com" created="Thu, 6 Jun 2013 11:58:28 +0000"  >&lt;p&gt;Secondary server&lt;/p&gt;</comment>
                            <comment id="354385" author="david.verdejo@logitravel.com" created="Thu, 6 Jun 2013 11:57:24 +0000"  >&lt;p&gt;Primary server&lt;/p&gt;</comment>
                            <comment id="354384" author="david.verdejo@logitravel.com" created="Thu, 6 Jun 2013 11:54:58 +0000"  >&lt;p&gt;2 notes:&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;When I receive this message, I try to make a telnet connection to port 27017 to primary and it works fine.&lt;/li&gt;
	&lt;li&gt;To resolve this issue, I tried to stop the service and start but it was still failing. I stopped the service, deleted all files in dbpath and started the service and now it&apos;s working fine.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;I will send you the logs from the servers.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="132661">SERVER-13681</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="133203">SERVER-13725</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="133228">SERVER-13729</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="114461">SERVER-12880</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="128814">SERVER-13444</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="27667" name="mongo_logmngsc11.zip" size="4957198" author="david.verdejo@logitravel.com" created="Thu, 6 Jun 2013 11:57:24 +0000"/>
                            <attachment id="27668" name="mongo_logmngsc21.zip" size="209860" author="david.verdejo@logitravel.com" created="Thu, 6 Jun 2013 11:58:28 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>11.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 18 Jul 2013 16:42:13 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        9 years, 45 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-13681'>SERVER-13681</a></s>, <s><a href='https://jira.mongodb.org/browse/SERVER-13729'>SERVER-13729</a></s>, <s><a href='https://jira.mongodb.org/browse/SERVER-13725'>SERVER-13725</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-8</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            9 years, 45 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10022"><![CDATA[Windows]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>dan@mongodb.com</customfieldvalue>
            <customfieldvalue>david.verdejo@logitravel.com</customfieldvalue>
            <customfieldvalue>byteofthat</customfieldvalue>
            <customfieldvalue>mark.benvenuto@mongodb.com</customfieldvalue>
            <customfieldvalue>sam.helman@10gen.com</customfieldvalue>
            <customfieldvalue>sflint@sailthru.com</customfieldvalue>
            <customfieldvalue>sidd@tinyurl.com</customfieldvalue>
            <customfieldvalue>thomas.rueckstiess@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrmqvz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrg2un:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7364</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsimm7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>