[SERVER-10363] User should be able to specify 2dsphere paramaters Created: 28/Jul/13  Updated: 06/Dec/22  Resolved: 12/Sep/17

Status: Closed
Project: Core Server
Component/s: Geo
Affects Version/s: 2.4.0
Fix Version/s: 2.6.0

Type: Improvement Priority: Minor - P4
Reporter: Stephen Remde Assignee: Backlog - Query Team (Inactive)
Resolution: Done Votes: 2
Labels: indexing
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

all


Issue Links:
Duplicate
is duplicated by SERVER-10451 2dsphere default parameters are poor Closed
Related
related to SERVER-50172 Allow maxCellsInCovering index parameter Backlog
is related to SERVER-18724 findOne with $near force scan whole c... Closed
is related to SERVER-30915 Creating 2dsphere index on collection... Closed
Assigned Teams:
Query
Backwards Compatibility: Fully Compatible
Participants:

 Description   

My database has approximately 103 million documents representing polygons in the UK.

Currently, a query for a 10x10m square can scan 500-100 rows and return 5.

I know S2-geo-lib has the potential for cells of <2cm so I believe the default values in S2IndexingParams:

maxCellsInCovering
finestIndexedLevel
coarsestIndexedLevel

are inadequate for me at the moment, and so it would be beneficial to be able to set these upon index creation.



 Comments   
Comment by Asya Kamsky [ 12/Sep/17 ]

This seems to have been fixed around version 2.5.5.

Comment by Stephen Remde [ 20/Jan/16 ]

Dan Pasette - I think this can be closed now as finestIndexedLevel and coarsestIndexedLevel were already settable (but undocumented) and maxCellsInCovering is now implemented in SERVER-19168.

Comment by Kevin Albertson [ 24/Jul/15 ]

SERVER-19168 introduces maxCellsInCovering as an indexing parameter. Additionally, the default max cells for the next version of the 2dsphere index is lowered from 50 to 20.

Comment by Daniel Pasette (Inactive) [ 09/Mar/14 ]

This task is just missing maxCellsInCovering. finestIndexedLevel and coarsestIndexedLevel are settable at index creation time.

Comment by Stephen Remde [ 18/Feb/14 ]

Hari Khalsa,
Thanks, this would be great. And thanks for confirming these parameters are not use in queries.

Chris Lambert, see
https://docs.google.com/presentation/d/1Hl4KapfAENAOf4gv-pSngKwvS_jwNVHRPZTTDzXXn6Q/edit#slide=id.i122

Comment by Githook User [ 06/Jan/14 ]

Author:

{u'username': u'hkhalsa', u'name': u'Hari Khalsa', u'email': u'hkhalsa@10gen.com'}

Message: SERVER-10363 SERVER-11387 pay attention to special geo idx params
Branch: master
https://github.com/mongodb/mongo/commit/e89dccbef9b83ac8101dcceab64f5c6e09b6de50

Comment by hari.khalsa@10gen.com [ 24/Nov/13 ]

clambert@lyft.com Correct, the indexing and query code in the 2.5.x branch is in a state of flux and currently the extra 2dsphere parameters are ignored. I will fix it soon in 2.5.x. It should work in 2.4.x though.

Comment by Chris Lambert [ 24/Nov/13 ]

I overwrote coarsestIndexedLevel and finestIndexedLevel, but I think they're not used when creating the s2 criteria for a querying.

If I issue a very large (by coverage area) $geoWithin, no documents are returned unless they lie on the edges of my polygon. This made me think it was using the default value for coarsestIndexedLevel when covering the polygon for index, rather than the one specified in the index. Line 51 seems to confirm my suspicions, but I don't know the code enough to be sure:

https://github.com/mongodb/mongo/blob/ab167c642f49206b4328882286cd5b83c19088bd/src/mongo/db/index/expression_index.h#L51

Comment by hari.khalsa@10gen.com [ 18/Nov/13 ]

clambert
1. They're safe to use. I think they may currently be broken in 2.5.x but I should fix that shortly.

2. Defaults:
int coarsestIndexedLevel = S2::kAvgEdge.GetClosestLevel(100 * 1000.0 / kRadiusOfEarthInMeters);
int finestIndexedLevel = S2::kAvgEdge.GetClosestLevel(500.0 / kRadiusOfEarthInMeters);

So, should be ~ 500m and 100km on a side. The idea was to be able to index continents but also search for things in a city.

3. Not that I know of.

smremde I can make maxCells configurable. It should be in a 2.5.x. S2 uses it as an advisory parameter but hopefully it helps solve your problem.

Comment by Chris Lambert [ 12/Nov/13 ]

We're considering rolling our own s2/geohash based indexing solution since we can't seem to get efficient performance from Mongo's built in geospatial indexing, even with the new 2dsphere index type. I just found this ticket, though, and tuning finestIndexedLevel and coarsestIndexedLevel seems like it would help substantially. Before going down this road, I'd love to know:

1) Are these index options safe to use? They don't seem to be documented anywhere, so I want to make sure we can use them safely without support being pulled from a future version.
2) What are the default values? We obviously want to tune for the right tradeoff for our indexed geometry & query areas, but I'd like to know what the defaults are to start with. as best as I can tell from reading the code, the finest level represents ~8 meter s2 cells, and the coarsest represents ~1.8km. Is this correct?
3) Is there a good page that lists s2cell levels and approximate coverage areas? I know Mongo's just using the open source S2 library, but I was wondering if anyone found a cheat sheet.

Comment by Stephen Remde [ 07/Aug/13 ]

So, from poking around the code, I found it is possible to set finestIndexedLevel and coarsestIndexedLevel, via the index options:

db.collection.ensureIndex(

{geometry:"2dsphere"}

,

{finestIndexedLevel:30, coarsestIndexedLevel:10}

)

This is enough for me IFF the default value for maxCellsInCovering was sensible. It is not.

Generated at Thu Feb 08 03:22:58 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.