<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:59:45 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-42160] $group stages that use a DISTINCT_SCAN do not use a SHARDING_FILTER on sharded collections</title>
                <link>https://jira.mongodb.org/browse/SERVER-42160</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;$group stages that leverage a DISTINCT_SCAN in their execution do not&#160;produce a SHARDING_FILTER stage in their query plans. This presents a problem when orphan documents persist at entry to such a stage and potentially effect the result of the $group.&lt;/p&gt;

&lt;p&gt;Interestingly, because $groups using a DISTINCT_SCAN need to examine only one document for each value of the group key to execute, the problem is generally hidden; such $group&apos;s naturally &quot;deduplicate&quot; the orphan documents by selecting only one document for each value of the group key. However, there are (at least) three cases where this still presents problems: &lt;br/&gt;
 1) If documents are updated before a $group using a DISTINCT_SCAN runs, and before orphan documents are otherwise purged, there is a potential for the $group to use and pass forward stale values that it takes from the orphan documents rather than the &quot;live&quot;, updated ones. &lt;br/&gt;
 2) If &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-5477&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-5477&lt;/a&gt; is merged, then $groups using DISTINCT_SCANS will run exclusively on the shards when the $group keys superset the shard keys. This will exacerbate this issue because it means orphans won&apos;t be deduplicated &quot;accidentally&quot; by the $group, as in this case the $group runs under the assumption that all documents with the same value for the group keys are on the same shard.&lt;/p&gt;

&lt;p&gt;3) If orphan documents persist after their &quot;parent&quot; documents have been deleted, the distinct scan can return stale values from these documents.&#160;&lt;/p&gt;</description>
                <environment></environment>
        <key id="852941">SERVER-42160</key>
            <summary>$group stages that use a DISTINCT_SCAN do not use a SHARDING_FILTER on sharded collections</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-query-optimization">Backlog - Query Optimization</assignee>
                                    <reporter username="george.wangensteen@mongodb.com">George Wangensteen</reporter>
                        <labels>
                            <label>qexec-team</label>
                            <label>query-44-grooming</label>
                    </labels>
                <created>Thu, 11 Jul 2019 14:21:03 +0000</created>
                <updated>Mon, 24 Jul 2023 07:35:08 +0000</updated>
                                                                            <component>Aggregation Framework</component>
                                        <votes>0</votes>
                                    <watches>19</watches>
                                                                                                                <comments>
                            <comment id="4395202" author="charlie.swanson" created="Mon, 7 Mar 2022 15:48:25 +0000"  >&lt;p&gt;After discussion, we think that some of the solutions outlined above are worth pursuing, but they are expensive enough that we won&apos;t schedule this ticket on its own. With a fix on the horizon, we&apos;re not interested in risking a performance change to fix this bug at the moment. &lt;/p&gt;

&lt;p&gt;After confirming that some of the correct use cases (e.g. using shard key in the index) are still working, I&apos;m moving this ticket back to the backlog and linking it to a future project to improve grouping performance on sharded clusters where this work will fit in well.&lt;/p&gt;</comment>
                            <comment id="4354667" author="charlie.swanson" created="Mon, 14 Feb 2022 21:48:59 +0000"  >&lt;p&gt;An update to this ticket after picking it up and dusting it off:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;The consensus seems to be that in order to fix this ticket, we need to do better than just preventing DISTINCT_SCAN optimization when a SHARD_FILTER stage would also need to be present. To be clear, that is an option engineering wise and would fix the correctness issue, but would result in much slower plans for these types of queries. The two stages are currently incompatible from a correctness perspective &lt;em&gt;except&lt;/em&gt; when the distinct key is exactly the shard key. There is no existing logic to specifically look for this overlap. So, in approximate order of increasing complexity, we have these ideas:
	&lt;ol&gt;
		&lt;li&gt;Do nothing (maintains performance characteristics but leaves an outstanding correctness issue). Probably we would schedule a project later to do one of the more ambitious things below, which should get easier with time in the new optimizer.&lt;/li&gt;
		&lt;li&gt;Allow DISTINCT_SCAN and SHARD_FILTER on a single field which is exactly the shard key. &lt;font color=&quot;#00875a&quot;&gt;(easy)&lt;/font&gt;&lt;/li&gt;
		&lt;li&gt;Option to think about and expand for dotted cases? e.g. shard key is &quot;a.b&quot; distinct on &quot;a&quot;? &lt;font color=&quot;#00875a&quot;&gt;(easy-ish? More nuanced)&lt;/font&gt;&lt;/li&gt;
		&lt;li&gt;Add logic to permit DISTINCT_SCAN + SHARD_FILTER in cases where the index has both the distinct key &lt;em&gt;and&lt;/em&gt; &lt;b&gt;all&lt;/b&gt; shard key fields, but the two can be different. We think this would require some decent amount of new work. Either:
		&lt;ol&gt;
			&lt;li&gt;The DISTINCT_SCAN stage would have to be modified and integrated with the SHARD_FILTER stage (&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=ian.boros&quot; class=&quot;user-hover&quot; rel=&quot;ian.boros&quot;&gt;ian.boros&lt;/a&gt; and I were thinking of making a more generic &quot;extra predicate&quot; to be applied while doing the scan) &lt;font color=&quot;#ffab00&quot;&gt;(easy to medium, but not great architecturally and risky performance-wise)&lt;/font&gt;&lt;/li&gt;
			&lt;li&gt;The DISTINCT_SCAN would need to distinct on the entire index key. This would make SHARD_FILTER correct, but would produce non-unique keys which would require further distinct-ing. Because it is producing one key per unique shard key, it may also be substantially less efficient. In cases where the distinct key is the prefix, this could work pretty well in theory, but it may be tricky to always improve plan quality with this kind of strategy. &lt;font color=&quot;#ff8b00&quot;&gt;(medium, risky performance-wise)&lt;/font&gt;
			&lt;ul&gt;
				&lt;li&gt;Note that it appears that the distinct command is capable of post-processing to de-duplicate, but the aggregation pipeline would need some tweaking.&lt;/li&gt;
			&lt;/ul&gt;
			&lt;/li&gt;
		&lt;/ol&gt;
		&lt;/li&gt;
		&lt;li&gt;Add a fully generic mechanism to allow the combination of DISTINCT_SCAN and SHARD_FILTER in any scenario (notably now including those requiring a FETCH, where the shard key is not in the DISTINCT_SCAN index). This seems possible but would require building a pretty complex query tree. The idea would be that DISTINCT_SCAN will produce all distinct keys which we know is a superset of the correct answer. When we would normally apply a SHARD_FILTER we could double-check via a sub-query (sub-pipeline? sub-plan? something) to see if there is a non-orphaned document which contains that key. If we would have filtered out the one value, but there is a version of that key which &lt;em&gt;does&lt;/em&gt; belong on this shard, we can keep that value. I think this would work pretty well if there is an index available on both the distinct key and the shard key for quick lookup, or if the shard key is relatively unique (and we have a shard key index for sure), but may require a hash join to be a palatable solution in general, and even then I would question a heuristic to use DISTINCT_SCAN. &lt;font color=&quot;#de350b&quot;&gt;(hard)&lt;/font&gt;&lt;/li&gt;
	&lt;/ol&gt;
	&lt;/li&gt;
	&lt;li&gt;&lt;font color=&quot;#172b4d&quot;&gt;Note that for the cases not &quot;fixed&quot; by any bullet, we have the option of leaving the existing incorrect case, or disallowing DISTINCT_SCAN. I was imagining disallowing DISTINCT_SCAN so that the correctness problem disappears, but that does leave a risk of substantial performance regression for the cases which are not called out and fixed. Option 5 seems like the best hope of fixing everything and maintaining performance, but it unfortunately doesn&apos;t seem like we could guarantee a performance win over a non-DISTINCT_SCAN alternative.&lt;/font&gt;&lt;/li&gt;
&lt;/ul&gt;


&lt;ul&gt;
	&lt;li&gt;#4 and #5 would potentially build some complex query plans. This is not necessarily all that scary, but it would create some serious complexity or tension around duplicating or combining operators from PlanStages and DocumentSources. We know we want PlanStages for the DISTINCT_SCAN logic, but some existing operators like DocumentSourceLookup or DocumentSourceInternalShardFilter may be handy in combination. Unfortunately, we&apos;re not well positioned to build a hybrid tree like this without leading to a crazy nesting doll type plan where we go PlanStages within DocumentSources within a PlanStage within DocumentSources... doesn&apos;t sound fun. There are some other ideas which I won&apos;t record here for brevity.&lt;/li&gt;
&lt;/ul&gt;


&lt;ul&gt;
	&lt;li&gt;Because of that, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=ian.boros&quot; class=&quot;user-hover&quot; rel=&quot;ian.boros&quot;&gt;ian.boros&lt;/a&gt; and I were considering some options for #4 and #5 which would be built exclusively in SBE-land. This would allow us to compose operators that are all in the same &quot;space&quot; (uniformly SBE), but would probably necessitate some fairly substantial new planning logic in the query planner immediately around SBE. It&apos;s my understanding that the SBE world doesn&apos;t consider DISTINCT_SCAN plans today, and the logic for $group pushdown is helpful but not extremely well positioned to solve this type of problem. So we&apos;d need all the logic like &lt;a href=&quot;https://github.com/mongodb/mongo/blob/53d7bceee61f73a1d6959edb5d490c3b338f3c0d/src/mongo/db/query/get_executor.cpp#L2031-L2061&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this&lt;/a&gt; to rule out inapplicable indexes, and probably more complex logic if we want to build upon that for options #3 or #4.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;I&apos;m going to explore option 2 and option #4.1 since they should be relatively simple to test.&lt;/p&gt;

&lt;p&gt;cc &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=justin.seyster&quot; class=&quot;user-hover&quot; rel=&quot;justin.seyster&quot;&gt;justin.seyster&lt;/a&gt; since you provided the most recent patch for this which just banned DISTINCT_SCAN in cases where SHARD_FILTER would make it incorrect. cc &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=asya&quot; class=&quot;user-hover&quot; rel=&quot;asya&quot;&gt;asya&lt;/a&gt;&#160;since you were part of the decision crew for &quot;we need to do better than just preventing DISTINCT_SCAN optimization when a SHARD_FILTER stage would also need to be present&quot;&lt;/p&gt;</comment>
                            <comment id="2406336" author="justin.seyster" created="Tue, 3 Sep 2019 23:26:40 +0000"  >&lt;p&gt;Update on this: The fix is triggering some test failures, which I plan to diagnose before sending to code review.&lt;/p&gt;</comment>
                            <comment id="2400676" author="justin.seyster" created="Thu, 29 Aug 2019 22:54:53 +0000"  >&lt;p&gt;The most likely fix here is to ban DISTINCT_SCAN on shards when executing a sharded query, and I have some code written up that will do that. This change feels a little fraught, so I want to consider all the cases. Note that this will also affect the distinct command, which uses the same code path to generate its query plan.&lt;/p&gt;

&lt;p&gt;1) Distinct command on a sharded collection: I have an integration test written that gets an incorrect result from a distinct command, because it returns a result from an orphaned document. (I didn&apos;t look into it, but this but might have been around since before &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9507&quot; title=&quot;Optimize $sort+$group+$first pipeline to avoid full index scan&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9507&quot;&gt;&lt;del&gt;SERVER-9507&lt;/del&gt;&lt;/a&gt;.) With the fix in place, customers may see a loss in performance, because they will need to use an index scan or collection scan where a distinct scan was previously possible.&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;In general, it is not correct to attach a distinct scan to a shard filter, forcing us to ban the distinct scan.&lt;/li&gt;
	&lt;li&gt;However, we &lt;em&gt;could&lt;/em&gt; safely attach a distinct scan to a shard filter when the shard key is a prefix of the distinct field (e.g., distinct(&quot;a.b&quot;) when the shard key is &quot;a&quot;). My current fix &lt;em&gt;does not&lt;/em&gt; consider this case, and we will lose performance unnecessarily as a result. It&apos;s probably not worth the complexity, though, to address this case.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;2) Pipelines with $sort followed by $group: The pipeline gets split at the $sort stage before the &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9507&quot; title=&quot;Optimize $sort+$group+$first pipeline to avoid full index scan&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9507&quot;&gt;&lt;del&gt;SERVER-9507&lt;/del&gt;&lt;/a&gt; (use a distinct scan for $group when possible) examines the pipeline. The $group stage ends up in the merging part of the pipeline, so there is never a chance to use a distinct scan. This case will not be affected by my fix.&lt;br/&gt;
3) Pipelines with $group but without $sort: These pipelines get split in a way that sends a $group to the shard, and the shard may choose a distinct scan, even though no sort was requested (&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9507&quot; title=&quot;Optimize $sort+$group+$first pipeline to avoid full index scan&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9507&quot;&gt;&lt;del&gt;SERVER-9507&lt;/del&gt;&lt;/a&gt;). As with case 1, there is an obscure possibility of an incorrect result because of orphaned documents. Customers may see a performance loss as a result of the fix.&lt;br/&gt;
    -Also as with case 1, it would be possible to keep the distinct scan optimization in place when the shard key is a prefix of the group-by field, but I don&apos;t plan to check for that edge case.&lt;/p&gt;

&lt;p&gt;I&apos;m going to spend a little more time on testing and then put my proposed fix up for code review.&lt;/p&gt;</comment>
                            <comment id="2322075" author="justin.seyster" created="Thu, 11 Jul 2019 22:28:41 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=asya&quot; class=&quot;user-hover&quot; rel=&quot;asya&quot;&gt;asya&lt;/a&gt; This bug is possible in almost all cases when there are orphan documents living in shards targeted by the $group. If a shard finds an index that it can use with the distinct scan optimization, it generates a plan with no shard filter, and there is a possibility that it will return an orphan document to the merging node.&lt;/p&gt;

&lt;p&gt;To fix this bug, we will make the distinct scan optimization aware of the shard filter, but that will mean that shard filtering makes this filtering impossible in almost all cases (meaning it will only benefit non-sharded collections). The distinct scan is only compatible with the shard filter if the shard key is a prefix of the field we are grouping by.&lt;/p&gt;</comment>
                            <comment id="2321892" author="asya" created="Thu, 11 Jul 2019 20:39:19 +0000"  >&lt;p&gt;This is only the case when the distinct value is not on the shard key, right?&lt;/p&gt;</comment>
                            <comment id="2321585" author="george.wangensteen" created="Thu, 11 Jul 2019 18:15:46 +0000"  >&lt;p&gt;After a good conversation about this with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=ian.boros&quot; class=&quot;user-hover&quot; rel=&quot;ian.boros&quot;&gt;ian.boros&lt;/a&gt;, we&apos;ve come to the unfortunate conclusion that (as currently implemented) distinct scans are never guaranteed to be correct on sharded collections. Consider the case where we attempted to have a distinct scan perform a shard filter. If the distinct scan found an orphan document and filtered it, it would have no means to reach other documents with the same index key value as the orphan, so it would jump to the next index key value. This would result in incorrect results, and the only way to fix it would be to change the distinct scan interface so that it has the means to access more than one document with the same group key, and scan documents with the same group key value until it finds one that isn&apos;t an orphan. This, though, would make the distinct scan as hypothetically slow as an index scan, although in theory there shouldn&apos;t be too many orphan documents to make it so. In any case this will require a not-insignificant change to the implementation of distinct scans to fix.&#160;&lt;/p&gt;</comment>
                            <comment id="2321039" author="george.wangensteen" created="Thu, 11 Jul 2019 14:25:16 +0000"  >&lt;p&gt;CC &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=justin.seyster&quot; class=&quot;user-hover&quot; rel=&quot;justin.seyster&quot;&gt;justin.seyster&lt;/a&gt;&#160;I believe you&apos;re most familiar with $group using distinct scans from &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9507&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-9507&lt;/a&gt;&#160;so let me know if you think the description here can be improved.&#160;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="35285">SERVER-5477</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="119094">SERVER-13116</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="1649201">SERVER-55200</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>15.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25126"><![CDATA[Query Optimization]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 11 Jul 2019 20:39:19 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 48 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-3182</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>denis.grebennicov@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 48 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>asya.kamsky@mongodb.com</customfieldvalue>
            <customfieldvalue>backlog-query-optimization</customfieldvalue>
            <customfieldvalue>charlie.swanson@mongodb.com</customfieldvalue>
            <customfieldvalue>george.wangensteen@mongodb.com</customfieldvalue>
            <customfieldvalue>justin.seyster@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hvdnov:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr28w7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2881">Query 2019-07-29</customfieldvalue>
    <customfieldvalue id="2882">Query 2019-08-12</customfieldvalue>
    <customfieldvalue id="2883">Query 2019-08-26</customfieldvalue>
    <customfieldvalue id="2884">Query 2019-09-09</customfieldvalue>
    <customfieldvalue id="2885">Query 2019-09-23</customfieldvalue>
    <customfieldvalue id="2886">Query 2019-10-07</customfieldvalue>
    <customfieldvalue id="3283">Query 2019-10-21</customfieldvalue>
    <customfieldvalue id="3288">Query 2019-12-30</customfieldvalue>
    <customfieldvalue id="3496">Query 2020-03-23</customfieldvalue>
    <customfieldvalue id="3497">Query 2020-04-06</customfieldvalue>
    <customfieldvalue id="5267">QE 2021-10-18</customfieldvalue>
    <customfieldvalue id="5269">QE 2021-11-01</customfieldvalue>
    <customfieldvalue id="5271">QE 2021-11-15</customfieldvalue>
    <customfieldvalue id="5286">QO 2022-02-21</customfieldvalue>
    <customfieldvalue id="5288">QO 2022-03-07</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;Assuming d.data is a sharded collection with orphan documents&#160;&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;db.data.explain().aggregate({$group: {_id: &lt;/span&gt;&lt;span style=&quot;color: blue; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;&apos;$_id&apos;&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;, i: {$sum: &lt;/span&gt;&lt;span style=&quot;color: blue; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;&apos;$i&apos;&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;}}})&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;&lt;em&gt;The query plan returned here uses a sharding filter stage as this group cannot leverage a distinct scan&lt;/em&gt;&#160;&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;db.data.explain().aggregate({$group: {_id: &lt;/span&gt;&lt;span style=&quot;color: blue; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;&apos;$_id&apos;&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;, i: {$first: &lt;/span&gt;&lt;span style=&quot;color: blue; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;&apos;$i&apos;&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;}}})&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;&lt;em&gt;The query plan returned here does not use a sharding filter stage as this group can leverage a distinct scan&lt;/em&gt;&#160;&lt;/p&gt;

&lt;p&gt;This was determined mostly by code/query plan inspection as reproducing an error resulting from this is timing-dependent.&#160;&lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hvd9y7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>