[SERVER-2940] Double precision errors on linux 32 bit builds Created: 12/Apr/11 Updated: 12/Jul/16 Resolved: 06/May/11 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Geo |
| Affects Version/s: | None |
| Fix Version/s: | 1.9.1 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Greg Studer | Assignee: | Greg Studer |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
(Ubuntu?) linux g++ (Ubuntu/Linaro 4.4.4-14ubuntu5) 4.4.5 |
||
| Backwards Compatibility: | Fully Compatible |
| Operating System: | Linux |
| Participants: |
| Description |
|
The optimizations applied to point-comparison math for 2d queries in core.h result in incorrect results. Basically: return _y > p._y ? p._y + radius >= _y : _y + radius >= p._y; ... returns incorrect results on linux32, but the equivalent : volatile double sum = _y > p._y ? p._y + radius : _y + radius; ... returns the correct results. Volatile is required. From mathias, adding compiler flags: -mfpmath=sse and either -msse2 or -march=pentium4 ... solves the problem, but limits our target architectures. |
| Comments |
| Comment by auto [ 06/May/11 ] |
|
Author: {u'login': u'gregstuder', u'name': u'gregs', u'email': u'greg@10gen.com'}Message: no volatile reqd for 64-bit systems |
| Comment by auto [ 12/Apr/11 ] |
|
Author: {u'login': u'gregstuder', u'name': u'gregs', u'email': u'greg@10gen.com'}Message: fix for bad optimization on linux32 |