[SERVER-3925] Add .skip() equivalent for $geoNear dbcommand Created: 22/Sep/11  Updated: 05/May/14  Resolved: 27/Feb/14

Status: Closed
Project: Core Server
Component/s: Geo
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: David DeRemer Assignee: A. Jesse Jiryu Davis
Resolution: Duplicate Votes: 33
Labels: commands, geoNear, query, skip
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-8261 system for commands to return cursors Closed
depends on SERVER-9395 $near should have a $minDistance oper... Closed
depends on DOCS-1702 Example of geo paging with minDistance Closed
is depended on by SERVER-5074 server crash when geoNear result grow... Closed
Related
is related to DOCS-1702 Example of geo paging with minDistance Closed
Participants:

 Description   

Skipping doesn't appear to be implemented for $geoNear at present.

Need a way to skip records to page through results when using the $geoNear database command. Using a normal $near with find().skip(20) works but when we need to use the $geoNear command for one reason or another we need similar functionality.

Please refer to question posted on google groups: http://groups.google.com/group/mongodb-user/browse_thread/thread/d7aefcd76cc9981e/599dba8a1b07d7c0#599dba8a1b07d7c0



 Comments   
Comment by A. Jesse Jiryu Davis [ 27/Feb/14 ]

DOCS-1702 will demonstrate a higher-performance alternative, better than "skip."

Comment by A. Jesse Jiryu Davis [ 13/Jul/13 ]

A skip parameter to geoNear isn't needed. Applications can use aggregation queries as Asya showed, although this is inefficient for large skips. Better yet, applications can use minDistance (SERVER-9395) to efficiently page through results without a skip. Once we have an example in the MongoDB Manual we can close this ticket.

Comment by Asya Kamsky [ 25/Jun/13 ]

Starting with 2.4 Aggregation Framework added support for $geoNear stage. http://docs.mongodb.org/manual/reference/aggregation/geoNear/

Is there a reason not to use that - further pipeline stages can use sort, limit, skip, etc.

Example:

db.zips.aggregate({$geoNear: 
                   { near:[-122.4,37.79], 
                    distanceField:"distFromSF", 
                    limit:50, 
                    spherical:true, 
                    distanceMultiplier:3959,
                    includeLocs:"loc",
                    maxDistance:0.08
                   }
                  },
                  {skip:40},
                  {limit:5}
)
 
{
	"result" : [
		{
			"city" : "DALY CITY",
			"loc" : [
				-122.478015,
				37.678696
			],
			"pop" : 57354,
			"state" : "CA",
			"_id" : "94015",
			"distFromSF" : 8.79341224327427
		},
		{
			"city" : "ALBANY",
			"loc" : [
				-122.295394,
				37.890045
			],
			"pop" : 17333,
			"state" : "CA",
			"_id" : "94706",
			"distFromSF" : 8.964980599576903
		},
		{
			"city" : "PIEDMONT",
			"loc" : [
				-122.24191,
				37.84368
			],
			"pop" : 15763,
			"state" : "CA",
			"_id" : "94618",
			"distFromSF" : 9.392779852090646
		},
		{
			"city" : "BERKELEY",
			"loc" : [
				-122.249964,
				37.85711
			],
			"pop" : 11833,
			"state" : "CA",
			"_id" : "94705",
			"distFromSF" : 9.410798634771558
		},
		{
			"city" : "BERKELEY",
			"loc" : [
				-122.257048,
				37.866428
			],
			"pop" : 23551,
			"state" : "CA",
			"_id" : "94704",
			"distFromSF" : 9.421156835705629
		}
	],
	"ok" : 1
}

Comment by Jini [ 18/Jun/13 ]

We need skip in order to have a fine control over number of records returned for pagination purposes. Since other document stores in mongo support "skip", it only makes sense to have things consistent and provide the same functionality. It would be nice to have both skip and min distance.

Comment by Robert La Ferla [ 18/Jun/13 ]

How accurate is this "minDistance" going to be between queries? Will we lose results because of rounding or get duplicate results? I like the skip approach better. Which nosql store did you switch to?

Comment by hari.khalsa@10gen.com [ 18/Jun/13 ]

Hello. We are currently planning on adding a minDistance option to geoNear for 2dsphere indices in 2.6.0/2.5.x.

We think this is a better solution than adding a skip. Skip requires recomputing all distances and then discarding as many as you wish to skip. With minDistance, we know exactly where to restart the search and can avoid looking at any objects closer than minDistance.

Comment by Jini [ 18/Jun/13 ]

this issue will never be resolved seems like. We have since moved to another nosql store.

Comment by rob jennings [ 18/Jun/13 ]

this is also blocking my use of geoNear, resorting to near+skip, and haversine calcs in my app for distance

Comment by Robert La Ferla [ 24/Apr/13 ]

Thought: Instead of fixing geoNear (which is a run command), how about adding the "_distance" field as a dynamic field in response to a $near query? The $near find query returns a cursor so skip/limit/etc will all work. The essence is that the server is adding a field/attribute that doesn't exist in the record itself. It adds it dynamically. In order to differentiate dynamic fields from regular fields, they could be prefixed with an underscore. I would much prefer to see this because it gives us the results as a cursor so we can do more with it. It also introduces a new paradigm for Mongo that could be used elsewhere.

e.g.

db.mycollection.find({ loc:{$near:[-71.011297, 42.114022], $maxDistance:10}};

{ "_id" : ObjectId("5175ae221ec8e5d3620c5bcc"), "address" : "100 Main St", "country" : "US", "loc" : [ -71.105687, 42.308756 ], "city" : "Boston", "name" : "Mi Tierra Restaurant", "postcode" : "02130", "state" : "MA", "_distance": 0.002351 }

Thoughts???

Comment by Robert La Ferla [ 23/Apr/13 ]

Can we increase the priority on this issue? I see this as a major reason for using MongoDB and with no "workaround", I think upping the priority is merited.

Comment by Robert La Ferla [ 23/Apr/13 ]

This is a showstopper. Need skip for $geoNear.

Comment by Bill Fratner [ 04/Mar/13 ]

Can we please schedule this to be fixed? Geospatial queries without skip equivalent are not scaling for us to the point where we may need to drop Mongo as a solution. Please help on this matter. Thank You much!

Comment by Kim D. [ 26/Feb/13 ]

Same issue. Really need the skip equivalent. Blocker here too.

Comment by adamsecure [ 10/Feb/13 ]

Running into same issue as others here. Any tips will be appreciated.

Comment by Tony Jarvis [ 03/Feb/13 ]

Any work arounds for this? Thanks

Comment by Jini [ 01/Feb/13 ]

With lack of pagination of geolocated search results, this is a complete blocker.

Comment by Scott [ 04/Jan/13 ]

I hope this issue moves up quick. This is a blocker for our organization to use Mongo as most searches utilize geoNear and the ability to not being able to page results is causing severe performance issues.

Comment by Nick Gerner [ 24/Feb/12 ]

$near doesn't work in a sharded environment. We were making use of $near and a cursor to get very large result sets this way. When we sharded we migrated to geoNear, but now get errors, assert failures and mongod crashes when we push the num too high.

Comment by Michael Huntington [ 16/Dec/11 ]

+1 this.. I know find() & $near is a fall back but having the extra distance data is a major plus using $geoNear.

Comment by David DeRemer [ 22/Sep/11 ]

Best workaround I can think of is to increase num by the increment of the paging for each subsequent command and then splice the results. Any other ideas, please comment. Thanks.

Generated at Thu Feb 08 03:04:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.