[SERVER-19634] Floating point error of distance computation in geoNear Created: 28/Jul/15  Updated: 28/Dec/23

Status: Backlog
Project: Core Server
Component/s: Geo
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Brandon Zhang Assignee: Backlog - Query Integration
Resolution: Unresolved Votes: 0
Labels: qi-geo, query-44-grooming
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query Integration
Operating System: ALL
Steps To Reproduce:

Run mongo/jstests/noPassthroughWithMongod/geo_axis_aligned.js and log the distances of the documents returned. Some of these calculated distances will be greater than their actual distance due to floating point error.

Participants:

 Description   

Documents near or on the boundaries of a geoNear search are susceptible to being excluded because the distance calculations in geoNear can produce inaccurate results due to floating point error (i.e. 54.001 - 54 > 0.001). Currently, this is accounted for in flat queries for by slightly expanding the last search interval and filtering with the TwoDPtInAnnulusExpression, but it is unclear if this handles all cases. Also, it may not be correct to return the documents if the distance calculation returns a value over the maxDistance.


Generated at Thu Feb 08 03:51:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.