<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:51:13 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-83091] $or query can trigger an infinite loop during plan enumeration</title>
                <link>https://jira.mongodb.org/browse/SERVER-83091</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Certain $or queries can trigger an infinite loop in the &lt;tt&gt;PlanEnumerator&lt;/tt&gt;. While the plan space can be quite large for indexed $or queries, we have logic which is supposed to bail out of plan enumeration once a certain number of alternatives have been generated. This logic is not kicking in correctly.&lt;/p&gt;

&lt;p&gt;Note that this only affects the so-called &quot;lockstep $or&quot; enumeration order which was first implemented in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-50291&quot; title=&quot;Add query knob to enumerate $or children in a different order&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-50291&quot;&gt;&lt;del&gt;SERVER-50291&lt;/del&gt;&lt;/a&gt;. Lockstep $or enumeration is intended to make it more likely that the system generates and scores interesting indexed $or plans, in the end resulting in better access paths for indexed $or predicates. It was enabled by default in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-74893&quot; title=&quot;Change default enumeration strategy for planning $or queries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-74893&quot;&gt;&lt;del&gt;SERVER-74893&lt;/del&gt;&lt;/a&gt; for versions 7.1.0-rc0 and 7.0.2. Many customers are first getting exposed to this new default $or plan enumeration logic as they upgrade to 7.0.2 which made us aware of the bug. The issue can be worked around safely by disabling lockstep $or enumeration. This can be done on the command line, in a config file, or with a shell command such as the following:&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;db.adminCommand({setParameter: 1, internalQueryEnumerationPreferLockstepOrEnumeration: false});&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Note that the infinite loop does not include any interrupt points, so the affected queries are impossible to kill and can block the server from shutting down. It may also manifest as increased CPU activity, since the thread is spinning busily rather than deadlocked.&lt;/p&gt;

&lt;p&gt;As of this writing, I&apos;ve confirmed that 7.0.2 is affected, but I plan to double-check whether more recent versions are affected by the bug as well.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2495984">SERVER-83091</key>
            <summary>$or query can trigger an infinite loop during plan enumeration</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="david.storch@mongodb.com">David Storch</assignee>
                                    <reporter username="david.storch@mongodb.com">David Storch</reporter>
                        <labels>
                            <label>bkp</label>
                    </labels>
                <created>Thu, 9 Nov 2023 22:28:55 +0000</created>
                <updated>Wed, 20 Dec 2023 18:13:48 +0000</updated>
                            <resolved>Tue, 28 Nov 2023 13:26:57 +0000</resolved>
                                    <version>7.0.2</version>
                    <version>7.0.3</version>
                    <version>7.2.0-rc1</version>
                                    <fixVersion>7.3.0-rc0</fixVersion>
                    <fixVersion>7.2.0-rc2</fixVersion>
                    <fixVersion>7.0.5</fixVersion>
                    <fixVersion>6.0.13</fixVersion>
                    <fixVersion>5.0.24</fixVersion>
                                                        <votes>0</votes>
                                    <watches>22</watches>
                                                                                                                <comments>
                            <comment id="5919324" author="xgen-internal-githook" created="Fri, 1 Dec 2023 20:20:32 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;David Storch&apos;, &apos;email&apos;: &apos;david.storch@mongodb.com&apos;, &apos;username&apos;: &apos;dstorch&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-83091&quot; title=&quot;$or query can trigger an infinite loop during plan enumeration&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-83091&quot;&gt;&lt;del&gt;SERVER-83091&lt;/del&gt;&lt;/a&gt; Fix infinite loop in lockstep $or plan enumeration&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 0b8d487a19477c72a64676db203b84763a363bbe)&lt;/p&gt;

&lt;p&gt;GitOrigin-RevId: 6dceb9b856961509390e3ff06f035eb65d81d74f&lt;br/&gt;
Branch: v5.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/23ed09bed9fbf327117205d7c8d54f963d51847a&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/23ed09bed9fbf327117205d7c8d54f963d51847a&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5918733" author="xgen-internal-githook" created="Fri, 1 Dec 2023 17:02:14 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;David Storch&apos;, &apos;email&apos;: &apos;david.storch@mongodb.com&apos;, &apos;username&apos;: &apos;dstorch&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-83091&quot; title=&quot;$or query can trigger an infinite loop during plan enumeration&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-83091&quot;&gt;&lt;del&gt;SERVER-83091&lt;/del&gt;&lt;/a&gt; Fix infinite loop in lockstep $or plan enumeration&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 479bb27e76bff47398a7c1d3060e818645cef8ed)&lt;br/&gt;
Branch: v6.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/0b8d487a19477c72a64676db203b84763a363bbe&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/0b8d487a19477c72a64676db203b84763a363bbe&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5910742" author="xgen-internal-githook" created="Tue, 28 Nov 2023 21:58:55 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;David Storch&apos;, &apos;email&apos;: &apos;david.storch@mongodb.com&apos;, &apos;username&apos;: &apos;dstorch&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-83091&quot; title=&quot;$or query can trigger an infinite loop during plan enumeration&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-83091&quot;&gt;&lt;del&gt;SERVER-83091&lt;/del&gt;&lt;/a&gt; Fix infinite loop in lockstep $or plan enumeration&lt;/p&gt;

&lt;p&gt;(cherry picked from commit f84e9c8325f3575929accc645e857c8d00c93391)&lt;br/&gt;
Branch: v7.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/479bb27e76bff47398a7c1d3060e818645cef8ed&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/479bb27e76bff47398a7c1d3060e818645cef8ed&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5909692" author="xgen-internal-githook" created="Tue, 28 Nov 2023 17:26:19 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;David Storch&apos;, &apos;email&apos;: &apos;david.storch@mongodb.com&apos;, &apos;username&apos;: &apos;dstorch&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-83091&quot; title=&quot;$or query can trigger an infinite loop during plan enumeration&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-83091&quot;&gt;&lt;del&gt;SERVER-83091&lt;/del&gt;&lt;/a&gt; Fix infinite loop in lockstep $or plan enumeration&lt;/p&gt;

&lt;p&gt;(cherry picked from commit a36f67db89cda0f0b9fbe54a97290a21616efd5d)&lt;br/&gt;
Branch: v7.2&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/f84e9c8325f3575929accc645e857c8d00c93391&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/f84e9c8325f3575929accc645e857c8d00c93391&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5907909" author="xgen-internal-githook" created="Tue, 28 Nov 2023 02:28:59 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;David Storch&apos;, &apos;email&apos;: &apos;david.storch@mongodb.com&apos;, &apos;username&apos;: &apos;dstorch&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-83091&quot; title=&quot;$or query can trigger an infinite loop during plan enumeration&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-83091&quot;&gt;&lt;del&gt;SERVER-83091&lt;/del&gt;&lt;/a&gt; Fix infinite loop in lockstep $or plan enumeration&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/a36f67db89cda0f0b9fbe54a97290a21616efd5d&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/a36f67db89cda0f0b9fbe54a97290a21616efd5d&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5877423" author="david.storch" created="Mon, 13 Nov 2023 23:46:00 +0000"  >&lt;p&gt;I&apos;ve solved the mystery of why this initially appeared as though it did not affect the master branch. The reason is that the issue is related to nested $or predicates, but these get simplified by the new boolean simplification module which was recently enabled under &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-81630&quot; title=&quot;Enable Boolean expression simplifier&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-81630&quot;&gt;&lt;del&gt;SERVER-81630&lt;/del&gt;&lt;/a&gt;. If I disable this boolean simplification step with the following command, then I can easily reproduce the issue:&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;db.adminCommand({setParameter: 1, internalQueryEnableBooleanExpressionsSimplifier: false});&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Boolean simplification makes this issue much harder to reproduce, though it still seems theoretically possible to construct an example in which nested $or predicates do not get flattened (e.g. if boolean simplification doesn&apos;t kick in) that is subject to the bug. Moreover, the system shouldn&apos;t depend on boolean simplification to behave correctly. I&apos;m marking the most recent release (7.2.0-rc0) as an affected version. The fix should be implemented in the master branch and backported from there in the usual way.&lt;/p&gt;</comment>
                            <comment id="5871769" author="david.storch" created="Fri, 10 Nov 2023 18:08:22 +0000"  >&lt;p&gt;After digging through my repro in a debugger, I think I have a sense of what&apos;s going wrong. The situation is that the &lt;tt&gt;PlanEnumerator&lt;/tt&gt; memo data structure has two &lt;tt&gt;LockstepOrAssignment&lt;/tt&gt; nodes, where one refers to the other as a subnode. The symptom is that during a call &lt;tt&gt;PlanEnumerator::nextMemo()&lt;/tt&gt; to advance the memo state, we get stuck in the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/4cdac1e55c22b2aa66ad5ba3d47181f7c80d1e29/src/mongo/db/query/plan_enumerator.cpp#L1789-L1796&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this while() loop &lt;/a&gt; associated with the &lt;em&gt;uppermost&lt;/em&gt; node.&lt;/p&gt;

&lt;p&gt;I think the fundamental problem is that the lockstep enumeration algorithm assumes that each of its subnodes has a fixed number of enumeration states, and will consistently report when it has &quot;wrapped around&quot;. For instance, imagine that a subnode has three enumeration states. The lockstep algorithm will only work correctly if every third recursive call to &lt;tt&gt;nextMemo()&lt;/tt&gt; on this subnode returns true in order to indicate that it has wrapped around. When the subnode is itself a &lt;tt&gt;LockstepOrAssignment&lt;/tt&gt; node, this assumption is violated. In particular, it is violated because the &lt;tt&gt;LockstepOrAssignment&lt;/tt&gt; subnode will &lt;a href=&quot;https://github.com/mongodb/mongo/blob/4cdac1e55c22b2aa66ad5ba3d47181f7c80d1e29/src/mongo/db/query/plan_enumerator.cpp#L1848-L1855&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;start returning true on every &lt;tt&gt;nextMemo()&lt;/tt&gt; call as soon as it has reached &lt;tt&gt;internalQueryEnumerationMaxOrSolutions&lt;/tt&gt; limit &lt;/a&gt;(which has a default value of 10).&lt;/p&gt;

&lt;p&gt;I think that we should probably fix this by relaxing the lockstep enumeration algorithm to work correctly even if a subnode starts wrapping around sooner than it did on the first round of plan enumeration. That way we don&apos;t have to fundamentally change the property that each OR node in the memo only allows a maximum of 10 distinct plan enumerations.&lt;/p&gt;

&lt;p&gt;The next steps for this ticket are to:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Confirm that the bug affects the master branch, and assuming that it does, alter the repro so that it works against the master branch.&lt;/li&gt;
	&lt;li&gt;Look into the details of a fix.&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="5871367" author="david.storch" created="Fri, 10 Nov 2023 16:28:11 +0000"  >&lt;p&gt;My script reproduces the issue on versions between 7.0.2 and 7.0.4-rc0 (the most recent 7.0.x release as of this writing). I could not reproduce it in the same way on 7.0.0, 7.0.1, or 7.2.0-rc1. This may be happenstance, or it may be that the bug really only affects certain versions on the 7.0 branch. I&apos;ll update the &quot;affectsVersions&quot; field accordingly for now.&lt;/p&gt;

&lt;p&gt;EDIT: I realized that the bug does not affect 7.0.0 or 7.0.1 because lockstep enumeration was not enabled as the default behavior until 7.0.2 (see &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-74893&quot; title=&quot;Change default enumeration strategy for planning $or queries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-74893&quot;&gt;&lt;del&gt;SERVER-74893&lt;/del&gt;&lt;/a&gt;). If I enable lockstep enumeration explicitly, then I can reproduce the infinite loop against those versions. However, my observation stands that 7.2.0-rc1 does not appear to be affected, even when lockstep enumeration is enabled.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="11220">
                    <name>Fix</name>
                                            <outwardlinks description="fixes">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="2509934">SERVER-83602</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="1440948">SERVER-50291</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="2456234">SERVER-81630</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="2289144">SERVER-74893</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="26553"><![CDATA[v7.2]]></customfieldvalue>
    <customfieldvalue key="25578"><![CDATA[v7.0]]></customfieldvalue>
    <customfieldvalue key="23470"><![CDATA[v6.0]]></customfieldvalue>
    <customfieldvalue key="21777"><![CDATA[v5.0]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[5006R00001y7Md3QAE, 5006R00001y6zI0QAI]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 10 Nov 2023 20:31:05 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        9 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            9 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>14.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i2zrgn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i2hl78:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="7258">QO 2023-11-13</customfieldvalue>
    <customfieldvalue id="7260">QO 2023-11-27</customfieldvalue>
    <customfieldvalue id="7262">QO 2023-12-11</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i2zdlz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>