[SERVER-26193] conversion from double to decimal rounds 0.999999999999999 incorrectly Created: 20/Sep/16  Updated: 19/Nov/16  Resolved: 29/Sep/16

Status: Closed
Project: Core Server
Component/s: Shell
Affects Version/s: None
Fix Version/s: 3.4.0-rc0

Type: Bug Priority: Major - P3
Reporter: Geert Bosch Assignee: Geert Bosch
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

> x=NumberDecimal(0.999999999999999)
NumberDecimal("1.00000000000000")

Sprint: Integration 2016-10-10
Participants:

 Description   

When determining the number of digits to round to, the existing code first uses a possible overestimate of the decimal place to round at, and corrects it if the final number of digits is not 15. However, this is incorrect when the rounding causes the decade to change: 1.00000000000000 has the same number of digits as 0.999999999999999. So, first try rounding at the smaller decimal place instead.



 Comments   
Comment by Githook User [ 29/Sep/16 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-26193 Fix incorrect rounding in double to decimal conversion
Branch: master
https://github.com/mongodb/mongo/commit/251a6f637b425c726aa2899b6d3bbf3c2be8aa98

Generated at Thu Feb 08 04:11:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.