Replace assert.close() with separate functions for integer and floating point closeness

XMLWordPrintableJSON

    • Server Tooling & Methods
    • ALL
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      assert.close() is used in our code both to compare for equality floating point numbers to a degree of precision and to integer values, evaluating only the X most significant digits. We should instead have separate functions for the 2. The current implementation can lead to surprising results when looking at it from only one use case:

      > 
> assert.close(1, 2, '', 4)
2019-01-04T16:26:47.030-0500 E QUERY    [js] Error: 1 is not equal to 2 within 4 places, absolute error: 1, relative error: 0.5 :
doassert@src/mongo/shell/assert.js:20:14
assert.close@src/mongo/shell/assert.js:903:9
@(shell):1:1
> 
> // Result may be surprising to someone looking for a floating point precision check.
> assert.close(100001, 100002, '', 4)
> 
> assert.close(0.01, 0.02, '', 4)
2019-01-04T16:27:11.086-0500 E QUERY    [js] Error: 0.01 is not equal to 0.02 within 4 places, absolute error: 0.01, relative error: 0.5 :
doassert@src/mongo/shell/assert.js:20:14
assert.close@src/mongo/shell/assert.js:903:9
@(shell):1:1
> 
> assert.close(0.000001, 0.000002, '', 4)
> 
> // This fails a floating point precision check, but does not assert.
> assert.close(2000000.1, 2000000.2, '', 4)
>

        1. Screen Shot 2019-01-06 at 12.04.21 AM.png
          Screen Shot 2019-01-06 at 12.04.21 AM.png
          230 kB
        2. Screen Shot 2019-01-06 at 12.04.48 AM.png
          Screen Shot 2019-01-06 at 12.04.48 AM.png
          201 kB

            Assignee:
            Backlog - Server Tooling and Methods (STM) (Inactive)
            Reporter:
            James Wahlin
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: