<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:56:29 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-84974] Investigate $geoNear blocking sort</title>
                <link>https://jira.mongodb.org/browse/SERVER-84974</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We think $geoNear already has to do some in-memory sorting.&#160; (It scans progressively larger annuli from the index, but it must sort the points within each annulus.)&lt;/p&gt;

&lt;p&gt;Is this something we can expose easily, to allow $geoNear to run anywhere in a pipeline?&lt;/p&gt;</description>
                <environment></environment>
        <key id="1802510">SERVER-84974</key>
            <summary>Investigate $geoNear blocking sort</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="david.percy@mongodb.com">David Percy</assignee>
                                    <reporter username="david.percy@mongodb.com">David Percy</reporter>
                        <labels>
                    </labels>
                <created>Tue, 29 Jun 2021 17:01:09 +0000</created>
                <updated>Fri, 12 Jan 2024 02:58:27 +0000</updated>
                            <resolved>Thu, 22 Jul 2021 04:11:26 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="3950311" author="david.storch" created="Mon, 19 Jul 2021 18:50:40 +0000"  >&lt;blockquote&gt;
&lt;p&gt;I think you&apos;re talking about the $near and $nearSphere match expressions here? It&apos;s confusing because internally they&apos;re spelled GEO_NEAR. I agree it&apos;s weird for a MatchExpression to imply a sort. By $geoNear I meant the&#160;&lt;a href=&quot;https://docs.mongodb.com/manual/reference/operator/aggregation/geoNear/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;$geoNear aggregation stage&lt;/a&gt;, so it&apos;s not weird in the same way. Luckily $near and $nearSphere are not allowed in a $match stage.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;Ah yeah, I was talking about the match expressions. You are right that the agg stage does not have the same problem. One side note: I&apos;m pretty sure that $geoNear is &lt;em&gt;also&lt;/em&gt; a MatchExpression and is a synonym for $near/$nearSphere. It &lt;a href=&quot;https://github.com/mongodb/mongo/blob/ad1adfaeab66f5a1c1093b5f1f2bbdc3bd0b48f8/src/mongo/db/matcher/expression_parser.cpp#L2130&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;shows up here&lt;/a&gt; in the MatchExpression parser.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;I think we still want the user to be able to write $geoNear (to make time-series collections act as much like a regular collection as we can), so maybe we would rewrite a $geoNear to $geoWithin + $sort if we can&apos;t push it down.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;That makes sense to me. Internally geoNear is all about the actual runtime algorithm. But from the user&apos;s perspective, we should be able to convert it into a regular geo match expression followed by a sort.&lt;/p&gt;</comment>
                            <comment id="3926485" author="david.percy" created="Fri, 9 Jul 2021 22:00:14 +0000"  >&lt;p&gt;Thanks for the feedback!  It sounds like blocking or top-k sort is a plausible query plan, but the question is whether we should express it with a separate $sort stage or not.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;It&apos;s always been kinda weird that $geoNear is a match expression but implies a sort&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;I think you&apos;re talking about the $near and $nearSphere match expressions here?  It&apos;s confusing because internally they&apos;re spelled GEO_NEAR.  I agree it&apos;s weird for a MatchExpression to imply a sort.  By $geoNear I meant the &lt;a href=&quot;https://docs.mongodb.com/manual/reference/operator/aggregation/geoNear/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;$geoNear aggregation stage&lt;/a&gt;, so it&apos;s not weird in the same way.  Luckily $near and $nearSphere are not allowed in a $match stage.&lt;/p&gt;

&lt;p&gt;Your suggestion about $geoWithin + $sort is interesting.  I think we still want the user to be able to write $geoNear (to make time-series collections act as much like a regular collection as we can), so maybe we would rewrite a $geoNear to $geoWithin + $sort if we can&apos;t push it down.&lt;/p&gt;</comment>
                            <comment id="3925312" author="david.storch" created="Fri, 9 Jul 2021 14:29:22 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=david.percy&quot; class=&quot;user-hover&quot; rel=&quot;david.percy&quot;&gt;david.percy&lt;/a&gt; I don&apos;t know the exact history around why $geoNear was implemented in such a special way. What I can say is that the $geoNear operator has always been super tied to the specific algorithm used to implement it. Making it more of a logical operation rather than a way for the user to say &quot;search ever-larger annuli and sort by distance from the center within each annulus&quot; would be nice, but I&apos;m unsure if that&apos;s the easiest way for us to satisfy the use case. It&apos;s always been kinda weird that $geoNear is a match expression but implies a sort...&lt;/p&gt;

&lt;p&gt;I know that we have discussed something like &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-37043&quot; title=&quot;Allow multi ringed shapes to be used for $geoWithin.&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-37043&quot;&gt;SERVER-37043&lt;/a&gt; in the past, where we extend $geoWithin to be able to express ring shapes. And then we would have to fully expose a way to explicitly sort in the query by the geo distance metadata. That might prove to be an easier solution to implement, and it would get us away from the awkward match+sort semantics of $geoNear, as well as having to change the current position requirements around $geoNear.&lt;/p&gt;</comment>
                            <comment id="3922005" author="david.percy" created="Wed, 7 Jul 2021 23:28:45 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=david.storch&quot; class=&quot;user-hover&quot; rel=&quot;david.storch&quot;&gt;david.storch&lt;/a&gt; do you have any thoughts on this?&lt;/p&gt;

&lt;p&gt;I&apos;m assuming that the reason we never implemented this, is that sorting the whole collection would be a bad query plan.  And maybe geo collections tend to be big.  So we decided it&apos;s better to raise an error rather than pick such a bad plan.  Is this right?  Were there other reasons?&lt;/p&gt;

&lt;p&gt;If the pipeline has a limit then we could do a top-k sort.  And if the $geoNear has a maxDistance then a blocking sort might not be so bad, if the number of documents is small.&lt;/p&gt;

&lt;p&gt;I&apos;m interested in this because I think it would be much simpler than a progressive sort, for time-series measurements.  I&apos;d like $geoNear to be a normal stage, like $sort, that can appear anywhere in a pipeline.  Then we can do rewrites like:&lt;br/&gt;
[ $unpackBucket, $geoNear ]&lt;br/&gt;
=&amp;gt;&lt;br/&gt;
[ $geoIntersects, $unpackBucket, $geoNear ]&lt;/p&gt;

&lt;p&gt;The $geoIntersects would use an index to pick only a few buckets, then the $geoNear would do a (hopefully small) blocking sort.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="1821102">SERVER-58602</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1828035">SERVER-58745</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 9 Jul 2021 14:29:22 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 29 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-2232</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>kelly.lewis@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 29 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>david.percy@mongodb.com</customfieldvalue>
            <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzphif:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hz9x73:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4710">Query Optimization 2021-07-12</customfieldvalue>
    <customfieldvalue id="4712">Query Optimization 2021-07-26</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzp3rj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>