<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:43:09 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-57932] Improve error message for $near in aggregation to suggest workarounds</title>
                <link>https://jira.mongodb.org/browse/SERVER-57932</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Given the below description and comments, we&apos;ve repurposed this ticket into a request to improve the error message for $near. In particular, we would like the error to include information about how a $near predicate implies a sort of the data as well, which you may not want/need. If you don&apos;t want/need the sort part, there are workarounds available such as using $geoWithin predicates. These aren&apos;t easy workarounds, so we&apos;ll likely want to link to a docs page or something like that.&lt;/p&gt;
&lt;h3&gt;&lt;a name=&quot;OriginalDescription&quot;&gt;&lt;/a&gt;Original Description&lt;/h3&gt;

&lt;p&gt;Currently to perform a non-aggregation geo query one would use the $near operator. This operator is not allowed in aggregation pipeline, even as the first stage, and instead the aggregation pipeline requires a $geoNear stage.&lt;/p&gt;

&lt;p&gt;The rejection of $near by the aggregation pipeline causes particular problems because the drivers presently implement the count operation as an aggregation pipeline. Applications run into a problem when:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;They issue a find&lt;/li&gt;
	&lt;li&gt;In the find operation, they specify conditions which include $near&lt;/li&gt;
	&lt;li&gt;They want to count the results&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;The driver in this case wraps the query into an aggregation pipeline which the server subsequently fails.&lt;/p&gt;

&lt;p&gt;This was reported in &lt;a href=&quot;https://jira.mongodb.org/browse/MONGOID-5096&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/MONGOID-5096&lt;/a&gt;.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1792627">SERVER-57932</key>
            <summary>Improve error message for $near in aggregation to suggest workarounds</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="maddie.zechar@mongodb.com">Maddie Zechar</assignee>
                                    <reporter username="oleg.pudeyev@mongodb.com">Oleg Pudeyev</reporter>
                        <labels>
                            <label>neweng</label>
                    </labels>
                <created>Tue, 22 Jun 2021 13:27:26 +0000</created>
                <updated>Sun, 29 Oct 2023 21:51:48 +0000</updated>
                            <resolved>Mon, 6 Mar 2023 16:43:30 +0000</resolved>
                                                    <fixVersion>7.0.0-rc0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>9</watches>
                                                                                                                <comments>
                            <comment id="5252259" author="maddie.zechar" created="Mon, 6 Mar 2023 16:43:30 +0000"  >&lt;p&gt;do we want to backport this commit to earlier releases?&lt;/p&gt;</comment>
                            <comment id="5248733" author="xgen-internal-githook" created="Sat, 4 Mar 2023 02:24:41 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;Maddie Zechar&apos;, &apos;email&apos;: &apos;mez2113@columbia.edu&apos;, &apos;username&apos;: &apos;&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-57932&quot; title=&quot;Improve error message for $near in aggregation to suggest workarounds&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-57932&quot;&gt;&lt;del&gt;SERVER-57932&lt;/del&gt;&lt;/a&gt; Improve error message for $near in aggregation to suggest workarounds&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/1dddbcb90ec75240c29f04e00bd9486585bdb198&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/1dddbcb90ec75240c29f04e00bd9486585bdb198&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4788124" author="charlie.swanson" created="Tue, 30 Aug 2022 14:18:40 +0000"  >&lt;p&gt;I think so - yes. $near is only a MatchExpression operator - so the other contexts are somewhat more obscure. I believe maybe $graphLookup&apos;s &apos;restrictSearchWithMatch&apos;, and possibly one or two others? But the big one is definitely in a $match. If I recall correctly, the MatchExpression parser has some toggles for what features are allowed, so it would be fine to improve the error message in any context where $near is not allowed, depending on the phrasing of the new error message. &lt;/p&gt;</comment>
                            <comment id="4780277" author="maddie.zechar" created="Fri, 26 Aug 2022 18:19:00 +0000"  >&lt;p&gt;just want to confirm - we want this error message wherever/however a $near pops up in an agg pipeline? eg not just within a $match context like in the linked original ticket&lt;/p&gt;</comment>
                            <comment id="3978246" author="charlie.swanson" created="Tue, 3 Aug 2021 15:20:07 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=asya&quot; class=&quot;user-hover&quot; rel=&quot;asya&quot;&gt;asya&lt;/a&gt;&#160;I wanted to check if it was well understood what needs to change between $near and $geoWithin. I know that you can do $geoWithin inside a circle. Upon looking closer it looks like $near&apos;s min and max distance should let you compute a circle or a donut to use with $geoWithin. Those are optional though. Is $near&apos;s behavior to just return all documents if you don&apos;t specify those? So the translation of $near &lt;em&gt;without&lt;/em&gt; $minDistance or $maxDistance would just be to remove it?&#160;&lt;/p&gt;</comment>
                            <comment id="3969043" author="asya" created="Wed, 28 Jul 2021 19:36:47 +0000"  >&lt;p&gt;I don&apos;t have anything but I don&apos;t imagine it would be difficult - are you thinking of including some sort of reference to such an example in the error message?&#160; I think we usually only link to docs pages...&lt;/p&gt;</comment>
                            <comment id="3957529" author="charlie.swanson" created="Thu, 22 Jul 2021 14:36:51 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=asya&quot; class=&quot;user-hover&quot; rel=&quot;asya&quot;&gt;asya&lt;/a&gt;&#160;as discussed I converted this ticket into a request to improve the error. Do you have a blog post or any other resource that describes when and how you can translate a $near predicate into a $geoWithin?&#160;&lt;/p&gt;</comment>
                            <comment id="3898955" author="david.percy" created="Thu, 24 Jun 2021 21:05:59 +0000"  >&lt;p&gt;I was looking into a similar problem with views: we translate .find() on a view to .aggregate(). Since $match doesn&apos;t allow $near/$nearSphere, you can&apos;t use $near/$nearSphere on a view.&lt;/p&gt;

&lt;p&gt;I can see why it would be bad to allow $near and $nearSphere in a $match: that would mean $match can do both matching and sorting. We&apos;d have to reconsider all our $match rewrites, and be careful when adding new ones. I&apos;m guessing this is why $geoNear is its own stage.&lt;/p&gt;

&lt;p&gt;For views, we could probably translate $near/$nearSphere to a $geoNear stage. What&apos;s encouraging is we have some restrictions (in &lt;a href=&quot;https://github.com/mongodb/mongo/blob/4ebd3bc28ccef7b7e1ad91ce9d3f4338b61a7d0e/src/mongo/db/query/canonical_query.cpp#L369-L370&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;CanonicalQuery::isValid&lt;/a&gt;):&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Only one $near or $nearSphere is allowed per .find().&lt;/li&gt;
	&lt;li&gt;It must be at the top level, or within a top-level $and (after flattening $ands).&lt;/li&gt;
&lt;/ul&gt;


&lt;hr /&gt;
&lt;p&gt;For drivers, I see a few options:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;We drop support for $near/$nearSphere combined with .count(), and force users to use $geoWithin instead.&lt;/li&gt;
	&lt;li&gt;Drivers inspect the predicate and pull out $near/$nearSphere as a $geoNear stage. Basically reimplementing what the server would do for views.&lt;/li&gt;
	&lt;li&gt;The server could expose its find-to-aggregate translation logic somehow, to avoid each driver needing to reimplement it.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Imagine a new aggregation stage called $_internalFind, which accepts all the options that the find command does. Drivers could translate this:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;db.coll.find(filter, projection).sort(sort)&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;to this:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;db.coll.aggregate([ {$_internalFind: { filter, projection, sort }} ])&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;and let the server decide how to break that up into $geoNear, $match, $sort, etc.&lt;/p&gt;</comment>
                            <comment id="3897286" author="asya" created="Thu, 24 Jun 2021 14:28:28 +0000"  >&lt;p&gt;&amp;gt; rejection of $near by the aggregation pipeline causes particular problems because the drivers presently implement the count operation as an aggregation pipeline.&lt;/p&gt;

&lt;p&gt;This is puzzling because count doesn&apos;t make sense with &lt;tt&gt;$near&lt;/tt&gt; as it&apos;s a combination of geowithin plus sort.  For count user should be using $within or $geoWithin rather than $near and its analogues. &lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="2126011">DOCS-15596</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="1802504">SERVER-58443</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="1716431">MONGOID-5096</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 24 Jun 2021 14:03:21 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        48 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/DOCS-15596'>DOCS-15596</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-3064</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            48 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>asya.kamsky@mongodb.com</customfieldvalue>
            <customfieldvalue>charlie.swanson@mongodb.com</customfieldvalue>
            <customfieldvalue>david.percy@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>maddie.zechar@mongodb.com</customfieldvalue>
            <customfieldvalue>oleg.pudeyev@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hznt4n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr2at3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hznfdr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>