[SERVER-16007] Inconsistent rounding in float parser Created: 07/Nov/14  Updated: 04/Dec/14  Resolved: 04/Dec/14

Status: Closed
Project: Core Server
Component/s: JavaScript
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Jon Gorrono Assignee: Ramon Fernandez Marina
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Steps To Reproduce:

> var i = 1.2345
> parseFloat(i.toFixed(1))
1.2
> parseFloat(i.toFixed(2))
1.23
> parseFloat(i.toFixed(3))
1.234 <- should be a '5'
> parseFloat(i.toFixed(4))
1.2345
> parseFloat(i.toFixed(3))
1.234 <- should be a '5'
> var i = 1.234525
> parseFloat(i.toFixed(5))
1.23453 <- correct
> parseFloat(i.toFixed(4))
1.2345
> parseFloat(i.toFixed(3))
1.235 <- correct (mark as 'wtf')
> parseFloat(i.toFixed(1))
1.2
> parseFloat(i.toFixed(2))
1.23
> parseFloat(i.toFixed(3))
1.235 <- correct
> var i = 1.234525 (same variable, same value)
> parseFloat(i.toFixed(1))
1.2
> parseFloat(i.toFixed(2))
1.23
> parseFloat(i.toFixed(3))
1.235 <- correct
> var i = 1.22334455 (same variable, new value)
> parseFloat(i.toFixed(1))
1.2
> parseFloat(i.toFixed(2))
1.22
> parseFloat(i.toFixed(3))
1.223
> parseFloat(i.toFixed(4))
1.2233
> parseFloat(i.toFixed(5))
1.22334
> parseFloat(i.toFixed(6))
1.223345 <- correct
> parseFloat(i.toFixed(7))
1.2233445
> parseFloat(i.toFixed(1))
1.2
> parseFloat(i.toFixed(2))
1.22
> parseFloat(i.toFixed(3))
1.223
> parseFloat(i.toFixed(4))
1.2233
> parseFloat(i.toFixed(5))
1.22334
> parseFloat(i.toFixed(6))
1.223345 < -correct
> parseFloat(i.toFixed(7))
1.2233445
> parseFloat(i.toFixed(6))
1.223345 <-correct
> db.version()
2.6.5

Participants:

 Description   

I assume the server is handling EcmaScript parsing...

This is capricious and I am not sure if it is happening in parseFloat or in toFixed()

Some values are not being properly rounded. And sometimes the same value will be rounded differently on subsequent attemtps (see 'steps to reproduce').

It appears that repeating the rounding calculation produced an accurate result: first time does not.



 Comments   
Comment by Jon Gorrono [ 04/Dec/14 ]

Thanks, Ramon..... I'll do that.

Comment by Ramon Fernandez Marina [ 03/Dec/14 ]

Apologies for the late reply ccjon. Both toFixed() and parseInt() are implemented by the V8 shell, which is what the mongo shell is using. I took two cases from your repro and run them in Chrome's V8 shell, and I was able to reproduce this behavior:

> var i = 1.2345
> i.toFixed(3)
"1.234"
> var j = 1.234525
> j.toFixed(3)
"1.235"

I took a look at the spec for the toFixed() function, and there could be a bug; in step 8.a it reads:

Let n be an integer for which the exact mathematical value of n ÷ 10^f – x is as close to zero as possible. If there are two such n, pick the larger n.

Now x refers to our i, so for the case where x = 1.2345 this means:

1234/1000 - 1.2345 = -0.0005
1235/1000 - 1.2345 = +0.0005

The way I read the spec I understand that "n" should be 1235, but seems like 1234 is being picked instead. Perhaps the comparison is not on "n" but on "abs(n)", and when there are two identical "n" one of them is chosen arbitrarily?

Unless I'm missing something the issue appears to be within V8 itself, so I'd suggest you open a ticket in the V8 issue tracker.

Cheers,
Ramón.

Generated at Thu Feb 08 03:39:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.