<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:46:23 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-37574] Force reconfig should kill user operations</title>
                <link>https://jira.mongodb.org/browse/SERVER-37574</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Force reconfig from the command and from a heartbeat can lead to a stepdown and thus needs to kill user operations. &lt;br/&gt;
Here: &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a222ef5e647ac527f7d4f8636bcacd6cc0ae6b8e/src/mongo/db/repl/replication_coordinator_impl.cpp#L2329-L2335&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/a222ef5e647ac527f7d4f8636bcacd6cc0ae6b8e/src/mongo/db/repl/replication_coordinator_impl.cpp#L2329-L2335&lt;/a&gt;&lt;br/&gt;
And here: &lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/blob/a222ef5e647ac527f7d4f8636bcacd6cc0ae6b8e/src/mongo/db/repl/replication_coordinator_impl_heartbeat.cpp#L559-L568&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/a222ef5e647ac527f7d4f8636bcacd6cc0ae6b8e/src/mongo/db/repl/replication_coordinator_impl_heartbeat.cpp#L559-L568&lt;/a&gt;&lt;/p&gt;</description>
                <environment></environment>
        <key id="617120">SERVER-37574</key>
            <summary>Force reconfig should kill user operations</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="suganthi.mani@mongodb.com">Suganthi Mani</assignee>
                                    <reporter username="judah.schvimer@mongodb.com">Judah Schvimer</reporter>
                        <labels>
                    </labels>
                <created>Thu, 11 Oct 2018 19:28:27 +0000</created>
                <updated>Sun, 29 Oct 2023 22:27:28 +0000</updated>
                            <resolved>Mon, 20 May 2019 02:27:59 +0000</resolved>
                                                    <fixVersion>4.1.12</fixVersion>
                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="2251814" author="xgen-internal-githook" created="Mon, 20 May 2019 02:06:56 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Suganthi Mani&apos;, &apos;email&apos;: &apos;suganthi.mani@mongodb.com&apos;, &apos;username&apos;: &apos;smani87&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-37574&quot; title=&quot;Force reconfig should kill user operations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-37574&quot;&gt;&lt;del&gt;SERVER-37574&lt;/del&gt;&lt;/a&gt; Force reconfig should kill operations that conflict state&lt;br/&gt;
transition (primary to secondary/removed).&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/87c4ff3e8f939593c2ab0101f930648d8d208904&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/87c4ff3e8f939593c2ab0101f930648d8d208904&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2251811" author="suganthi.mani" created="Mon, 20 May 2019 02:05:13 +0000"  >&lt;p&gt;If the node has already step down due to reconfig code-path, then step down via heartbeat should simply return . Just noticed that when reading the code,&#160;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/8e6ad096f8a8b81e1be01d012920f52332650d6f/src/mongo/db/repl/replication_coordinator_impl_heartbeat.cpp#L411-L413&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;before returning&lt;/a&gt;, it should also &lt;a href=&quot;https://github.com/mongodb/mongo/blob/8e6ad096f8a8b81e1be01d012920f52332650d6f/src/mongo/db/repl/replication_coordinator_impl_heartbeat.cpp#L431&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;signal waiters&lt;/a&gt; waiting on the step down event. &lt;a href=&quot;https://github.com/mongodb/mongo/commit/8e6ad096f8a8b81e1be01d012920f52332650d6f&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/8e6ad096f8a8b81e1be01d012920f52332650d6f&lt;/a&gt;&#160;didn&apos;t address it. So, reverted the commit.&lt;/p&gt;</comment>
                            <comment id="2251808" author="xgen-internal-githook" created="Mon, 20 May 2019 01:54:08 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Suganthi Mani&apos;, &apos;email&apos;: &apos;suganthi.mani@mongodb.com&apos;, &apos;username&apos;: &apos;smani87&apos;}
&lt;p&gt;Message: Revert &quot;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-37574&quot; title=&quot;Force reconfig should kill user operations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-37574&quot;&gt;&lt;del&gt;SERVER-37574&lt;/del&gt;&lt;/a&gt; Force reconfig should kill operations that conflict state&quot;&lt;/p&gt;

&lt;p&gt;This reverts commit 8e6ad096f8a8b81e1be01d012920f52332650d6f.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/674276b0149d3b77b51ac46adc53b11c47a26519&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/674276b0149d3b77b51ac46adc53b11c47a26519&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2251738" author="xgen-internal-githook" created="Sun, 19 May 2019 21:15:07 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;email&apos;: &apos;suganthi.mani@mongodb.com&apos;, &apos;name&apos;: &apos;Suganthi Mani&apos;, &apos;username&apos;: &apos;smani87&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-37574&quot; title=&quot;Force reconfig should kill user operations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-37574&quot;&gt;&lt;del&gt;SERVER-37574&lt;/del&gt;&lt;/a&gt; Force reconfig should kill operations that conflict state&lt;br/&gt;
transition (primary to secondary/removed).&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/8e6ad096f8a8b81e1be01d012920f52332650d6f&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/8e6ad096f8a8b81e1be01d012920f52332650d6f&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2237027" author="suganthi.mani" created="Tue, 7 May 2019 14:40:57 +0000"  >&lt;p&gt;This ticket should handle below 2 things.&lt;/p&gt;

&lt;p&gt;1) &lt;a href=&quot;https://github.com/mongodb/mongo/blob/cf322abb3069b05e6ea32afe53a31381f0c0c6df/src/mongo/db/repl/replication_coordinator_impl_heartbeat.cpp#L405-L424&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Unconditional step down after acquiring mutex&lt;/a&gt; should check whether the member state is still primary and return if not. Else, it can lead to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/cf322abb3069b05e6ea32afe53a31381f0c0c6df/src/mongo/db/repl/topology_coordinator.cpp#L2376&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this invariant&lt;/a&gt; getting failed.&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;A concurrent step down might have happened via &lt;a href=&quot;https://github.com/mongodb/mongo/blob/cf322abb3069b05e6ea32afe53a31381f0c0c6df/src/mongo/db/repl/replication_coordinator_impl.cpp#L2629&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;reconfig&lt;/a&gt;/ reconfig via heartbeat, which means the leader mode is no longer in kSteppingDown.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;2) Reconfig cmd and reconfig via heartbeat should be able to take care of updating the term &amp;amp; resetting _pendingTermUpdateDuringStepDown (like we do it in &lt;a href=&quot;https://github.com/mongodb/mongo/blob/cf322abb3069b05e6ea32afe53a31381f0c0c6df/src/mongo/db/repl/replication_coordinator_impl_heartbeat.cpp#L426-L432.&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;unconditional step down&lt;/a&gt;) if they cause step down and _pendingTermUpdateDuringStepDown is set.&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Unconditional step down &lt;a href=&quot;https://github.com/mongodb/mongo/blob/cf322abb3069b05e6ea32afe53a31381f0c0c6df/src/mongo/db/repl/replication_coordinator_impl.cpp#L3742-L3748&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;triggered&lt;/a&gt;.&lt;/li&gt;
	&lt;li&gt;Reconfig gets executed and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/cf322abb3069b05e6ea32afe53a31381f0c0c6df/src/mongo/db/repl/topology_coordinator.cpp#L1909&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;caused the node to step down&lt;/a&gt;.&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/cf322abb3069b05e6ea32afe53a31381f0c0c6df/src/mongo/db/repl/replication_coordinator_impl_heartbeat.cpp#L380&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;ReplicationCoordinatorImpl::_stepDownFinish&lt;/a&gt; gets executed now.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;&lt;b&gt;This is a 4.0 bug and the fix must be backported.&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;&lt;b&gt;EDIT: Problem #2 will be addressed by&lt;/b&gt;&#160;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-41234&quot; title=&quot;Concurrent step downs due to step down via heartbeat and force reconfig  should ensure that the term maintained by topology coordinator is set to TopologyCoordinator::_pendingTermUpdateDuringStepDown.&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-41234&quot;&gt;&lt;del&gt;SERVER-41234&lt;/del&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&#160;CC &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2215600" author="ravind.kumar" created="Tue, 16 Apr 2019 19:25:03 +0000"  >&lt;p&gt;Note for docs - we should also update the following text for &lt;a href=&quot;https://docs.mongodb.com/manual/reference/command/replSetReconfig/#considerations&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;replSetReconfig&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;replSetReconfig obtains a special mutually exclusive lock to prevent more than one replSetReconfig operation from occurring at the same time.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;To be more specific. Via slack:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Reads and writes to admin database won&#8217;t be blocked because of the replication coordinator mutex lock. But if we take RSTL lock (the node steps down/removed because of reconfig), then the reads will be blocked during that period and write will be killed (after this ticket &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-37574&quot; title=&quot;Force reconfig should kill user operations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-37574&quot;&gt;&lt;del&gt;SERVER-37574&lt;/del&gt;&lt;/a&gt;)&lt;br/&gt;
 &#160;&lt;br/&gt;
 1) w/ &amp;amp; w/o &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-37574&quot; title=&quot;Force reconfig should kill user operations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-37574&quot;&gt;&lt;del&gt;SERVER-37574&lt;/del&gt;&lt;/a&gt; fix, during that period where RSTL lock is taken by reconfig we would be blocking the new incoming reads and writes. And, after that period, writes should be able to acquire the lock and it would be failing with NotMaster errorcode.&lt;/p&gt;

&lt;p&gt;2) W/o &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-37574&quot; title=&quot;Force reconfig should kill user operations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-37574&quot;&gt;&lt;del&gt;SERVER-37574&lt;/del&gt;&lt;/a&gt;, the in-progress read and write operations won&#8217;t be killed and we will allow it to complete when this reconfig is trying to acquire the RSTL lock. But after &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-37574&quot; title=&quot;Force reconfig should kill user operations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-37574&quot;&gt;&lt;del&gt;SERVER-37574&lt;/del&gt;&lt;/a&gt;, the in-progress write operations will be killed and read would be allowed to proceed.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;both of these cases result in retryable errors, and should be transparent to applications. Some of this information is not quite related to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-37574&quot; title=&quot;Force reconfig should kill user operations&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-37574&quot;&gt;&lt;del&gt;SERVER-37574&lt;/del&gt;&lt;/a&gt;, but since we&apos;re touch at least a bit on locking behavior and during what scenarios we trigger the mutex vs RSTL lock.&lt;/p&gt;</comment>
                            <comment id="2184790" author="judah.schvimer" created="Tue, 19 Mar 2019 14:15:50 +0000"  >&lt;p&gt;While I agree that we will not roll back w:majority writes that get acknowledged (since we close the connections), we could still write ordered operations out of order as described in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-27534&quot; title=&quot;All writing operations must fail if the term changes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-27534&quot;&gt;&lt;del&gt;SERVER-27534&lt;/del&gt;&lt;/a&gt;. I suspect that is almost as bad for some client applications.&lt;/p&gt;</comment>
                            <comment id="2184246" author="suganthi.mani" created="Mon, 18 Mar 2019 19:54:57 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;, in 4.0, we don&apos;t kill&#160; user operations on reconfig via command/heartbeat. But, we still &lt;a href=&quot;https://github.com/mongodb/mongo/blob/v4.0/src/mongo/db/repl/replication_coordinator_impl.cpp#L2612&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;close connections&lt;/a&gt;&#160;on step down. So, the current request result won&apos;t reach the client.&lt;/p&gt;

&lt;p&gt;I feel its good to backport&#160;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-27534&quot; title=&quot;All writing operations must fail if the term changes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-27534&quot;&gt;&lt;del&gt;SERVER-27534&lt;/del&gt;&lt;/a&gt; (kill user operations) to 4.0 just to make reconfig faster, but I don&apos;t see any correctness or some deadlock issue.According to me, the backport work is not that urgent.&lt;/p&gt;</comment>
                            <comment id="2184236" author="tess.avitabile" created="Mon, 18 Mar 2019 19:49:25 +0000"  >&lt;p&gt;Got it. In that case, we should backport this work.&lt;/p&gt;</comment>
                            <comment id="2184228" author="judah.schvimer" created="Mon, 18 Mar 2019 19:47:02 +0000"  >&lt;p&gt;I do think we need to backport it due to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-27534&quot; title=&quot;All writing operations must fail if the term changes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-27534&quot;&gt;&lt;del&gt;SERVER-27534&lt;/del&gt;&lt;/a&gt;. If we do not kill the operations and step down and then back up, when we check that we can accept writes, it will return true but we could have rolled back in the mean time. &lt;/p&gt;

&lt;p&gt;Alternatively we could check that the term hasn&apos;t changed since starting the operation in the OpObserver in logOp, and backport that which would maybe be easier.&lt;/p&gt;</comment>
                            <comment id="2184205" author="tess.avitabile" created="Mon, 18 Mar 2019 19:30:35 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=suganthi.mani&quot; class=&quot;user-hover&quot; rel=&quot;suganthi.mani&quot;&gt;suganthi.mani&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;, do you think we need to backport this change? I&apos;m not certain we need to. On older versions, we do not need to kill operations on stepdown in order to avoid deadlock; it just may take us longer to acquire the global lock. Additionally, it&apos;s okay if we don&apos;t kill write operations, since they will check if the node can accept writes after their next lock acquisition. I would prefer not to backport this if we don&apos;t need to, since the code has changed so much in this area.&lt;/p&gt;</comment>
                            <comment id="2181073" author="suganthi.mani" created="Thu, 14 Mar 2019 18:52:28 +0000"  >&lt;p&gt;Note: When this ticket is closed, need to file a DOC ticket to update &lt;a href=&quot;https://docs.mongodb.com/master/reference/method/rs.reconfig/#availability&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://docs.mongodb.com/manual/reference/method/rs.reconfig/#availability&lt;/a&gt; section&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;The rs.reconfig() shell method can trigger the current primary to step down in some situations. When the primary steps down, &lt;b&gt;it forcibly closes all client connections&lt;/b&gt;. Primary step-down triggers an election to select a new primary.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;&#160;After&#160;&#160;PM-639, we no longer close all client connections on primary step down.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="759654">SERVER-41037</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="730211">SERVER-40487</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10320">
                    <name>Documented</name>
                                                                <inwardlinks description="is documented by">
                                        <issuelink>
            <issuekey id="771821">DOCS-12717</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="759612">SERVER-41035</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="611497">SERVER-37381</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="341404">SERVER-27534</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="771806">SERVER-41234</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>12.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="15640"><![CDATA[v4.0]]></customfieldvalue>
    <customfieldvalue key="15141"><![CDATA[v3.6]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 14 Mar 2019 18:52:28 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 38 weeks, 3 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 38 weeks, 3 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>12.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>
            <customfieldvalue>ravind.kumar</customfieldvalue>
            <customfieldvalue>suganthi.mani@mongodb.com</customfieldvalue>
            <customfieldvalue>tess.avitabile@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hua173:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hu47yf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2542">Repl 2018-10-22</customfieldvalue>
    <customfieldvalue id="2822">Repl 2019-03-11</customfieldvalue>
    <customfieldvalue id="2823">Repl 2019-03-25</customfieldvalue>
    <customfieldvalue id="2896">Repl 2019-04-08</customfieldvalue>
    <customfieldvalue id="2918">Repl 2019-04-22</customfieldvalue>
    <customfieldvalue id="2919">Repl 2019-05-06</customfieldvalue>
    <customfieldvalue id="2920">Repl 2019-05-20</customfieldvalue>
    <customfieldvalue id="2999">Repl 2019-06-03</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu9ngf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>