[SERVER-2940] Double precision errors on linux 32 bit builds Created: 12/Apr/11  Updated: 12/Jul/16  Resolved: 06/May/11

Status: Closed
Project: Core Server
Component/s: Geo
Affects Version/s: None
Fix Version/s: 1.9.1

Type: Bug Priority: Major - P3
Reporter: Greg Studer Assignee: Greg Studer
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

(Ubuntu?) linux g++ (Ubuntu/Linaro 4.4.4-14ubuntu5) 4.4.5


Backwards Compatibility: Fully Compatible
Operating System: Linux
Participants:

 Description   

The optimizations applied to point-comparison math for 2d queries in core.h result in incorrect results. Basically:

return _y > p._y ? p._y + radius >= _y : _y + radius >= p._y;

... returns incorrect results on linux32, but the equivalent :

volatile double sum = _y > p._y ? p._y + radius : _y + radius;
return _y > p._y ? sum >= _y : sum >= p._y;

... returns the correct results. Volatile is required.

From mathias, adding compiler flags:

-mfpmath=sse and either -msse2 or -march=pentium4

... solves the problem, but limits our target architectures.



 Comments   
Comment by auto [ 06/May/11 ]

Author:

{u'login': u'gregstuder', u'name': u'gregs', u'email': u'greg@10gen.com'}

Message: no volatile reqd for 64-bit systems SERVER-2940
Branch: master
https://github.com/mongodb/mongo/commit/9c79d5735da100a25aed37c3a1339b69e8883c07

Comment by auto [ 12/Apr/11 ]

Author:

{u'login': u'gregstuder', u'name': u'gregs', u'email': u'greg@10gen.com'}

Message: fix for bad optimization on linux32 SERVER-2940
Branch: master
https://github.com/mongodb/mongo/commit/bba87960d05de77e29e002048fe28131bc703365

Generated at Thu Feb 08 03:01:37 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.