[SERVER-57932] Improve error message for $near in aggregation to suggest workarounds Created: 22/Jun/21  Updated: 29/Oct/23  Resolved: 06/Mar/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.0.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Oleg Pudeyev (Inactive) Assignee: Maddie Zechar
Resolution: Fixed Votes: 0
Labels: neweng
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on DOCS-15596 [Server] Add documentation for replac... Closed
Related
related to SERVER-58443 Allow $near/$nearSphere on a view Backlog
is related to MONGOID-5096 Calling "count" on query with $near r... Closed
Backwards Compatibility: Fully Compatible
Participants:

 Description   

Given the below description and comments, we've repurposed this ticket into a request to improve the error message for $near. In particular, we would like the error to include information about how a $near predicate implies a sort of the data as well, which you may not want/need. If you don't want/need the sort part, there are workarounds available such as using $geoWithin predicates. These aren't easy workarounds, so we'll likely want to link to a docs page or something like that.

Original Description

Currently to perform a non-aggregation geo query one would use the $near operator. This operator is not allowed in aggregation pipeline, even as the first stage, and instead the aggregation pipeline requires a $geoNear stage.

The rejection of $near by the aggregation pipeline causes particular problems because the drivers presently implement the count operation as an aggregation pipeline. Applications run into a problem when:

  • They issue a find
  • In the find operation, they specify conditions which include $near
  • They want to count the results

The driver in this case wraps the query into an aggregation pipeline which the server subsequently fails.

This was reported in https://jira.mongodb.org/browse/MONGOID-5096.



 Comments   
Comment by Maddie Zechar [ 06/Mar/23 ]

do we want to backport this commit to earlier releases?

Comment by Githook User [ 04/Mar/23 ]

Author:

{'name': 'Maddie Zechar', 'email': 'mez2113@columbia.edu', 'username': ''}

Message: SERVER-57932 Improve error message for $near in aggregation to suggest workarounds
Branch: master
https://github.com/mongodb/mongo/commit/1dddbcb90ec75240c29f04e00bd9486585bdb198

Comment by Charlie Swanson [ 30/Aug/22 ]

I think so - yes. $near is only a MatchExpression operator - so the other contexts are somewhat more obscure. I believe maybe $graphLookup's 'restrictSearchWithMatch', and possibly one or two others? But the big one is definitely in a $match. If I recall correctly, the MatchExpression parser has some toggles for what features are allowed, so it would be fine to improve the error message in any context where $near is not allowed, depending on the phrasing of the new error message.

Comment by Maddie Zechar [ 26/Aug/22 ]

just want to confirm - we want this error message wherever/however a $near pops up in an agg pipeline? eg not just within a $match context like in the linked original ticket

Comment by Charlie Swanson [ 03/Aug/21 ]

asya I wanted to check if it was well understood what needs to change between $near and $geoWithin. I know that you can do $geoWithin inside a circle. Upon looking closer it looks like $near's min and max distance should let you compute a circle or a donut to use with $geoWithin. Those are optional though. Is $near's behavior to just return all documents if you don't specify those? So the translation of $near without $minDistance or $maxDistance would just be to remove it? 

Comment by Asya Kamsky [ 28/Jul/21 ]

I don't have anything but I don't imagine it would be difficult - are you thinking of including some sort of reference to such an example in the error message?  I think we usually only link to docs pages...

Comment by Charlie Swanson [ 22/Jul/21 ]

asya as discussed I converted this ticket into a request to improve the error. Do you have a blog post or any other resource that describes when and how you can translate a $near predicate into a $geoWithin? 

Comment by David Percy [ 24/Jun/21 ]

I was looking into a similar problem with views: we translate .find() on a view to .aggregate(). Since $match doesn't allow $near/$nearSphere, you can't use $near/$nearSphere on a view.

I can see why it would be bad to allow $near and $nearSphere in a $match: that would mean $match can do both matching and sorting. We'd have to reconsider all our $match rewrites, and be careful when adding new ones. I'm guessing this is why $geoNear is its own stage.

For views, we could probably translate $near/$nearSphere to a $geoNear stage. What's encouraging is we have some restrictions (in CanonicalQuery::isValid):

  • Only one $near or $nearSphere is allowed per .find().
  • It must be at the top level, or within a top-level $and (after flattening $ands).

For drivers, I see a few options:

  • We drop support for $near/$nearSphere combined with .count(), and force users to use $geoWithin instead.
  • Drivers inspect the predicate and pull out $near/$nearSphere as a $geoNear stage. Basically reimplementing what the server would do for views.
  • The server could expose its find-to-aggregate translation logic somehow, to avoid each driver needing to reimplement it.

Imagine a new aggregation stage called $_internalFind, which accepts all the options that the find command does. Drivers could translate this:

db.coll.find(filter, projection).sort(sort)

to this:

db.coll.aggregate([ {$_internalFind: { filter, projection, sort }} ])

and let the server decide how to break that up into $geoNear, $match, $sort, etc.

Comment by Asya Kamsky [ 24/Jun/21 ]

> rejection of $near by the aggregation pipeline causes particular problems because the drivers presently implement the count operation as an aggregation pipeline.

This is puzzling because count doesn't make sense with $near as it's a combination of geowithin plus sort. For count user should be using $within or $geoWithin rather than $near and its analogues.

Generated at Thu Feb 08 05:43:09 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.