Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-2940

Double precision errors on linux 32 bit builds

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 1.9.1
    • Affects Version/s: None
    • Component/s: Geo
    • None
    • Environment:
      (Ubuntu?) linux g++ (Ubuntu/Linaro 4.4.4-14ubuntu5) 4.4.5
    • Fully Compatible
    • Linux

      The optimizations applied to point-comparison math for 2d queries in core.h result in incorrect results. Basically:

      return _y > p._y ? p._y + radius >= _y : _y + radius >= p._y;

      ... returns incorrect results on linux32, but the equivalent :

      volatile double sum = _y > p._y ? p._y + radius : _y + radius;
      return _y > p._y ? sum >= _y : sum >= p._y;

      ... returns the correct results. Volatile is required.

      From mathias, adding compiler flags:

      -mfpmath=sse and either -msse2 or -march=pentium4

      ... solves the problem, but limits our target architectures.

            Assignee:
            greg_10gen Greg Studer
            Reporter:
            greg_10gen Greg Studer
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved: