[SERVER-26193] conversion from double to decimal rounds 0.999999999999999 incorrectly Created: 20/Sep/16 Updated: 19/Nov/16 Resolved: 29/Sep/16 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Shell |
| Affects Version/s: | None |
| Fix Version/s: | 3.4.0-rc0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Geert Bosch | Assignee: | Geert Bosch |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Backwards Compatibility: | Fully Compatible |
| Operating System: | ALL |
| Steps To Reproduce: | > x=NumberDecimal(0.999999999999999) |
| Sprint: | Integration 2016-10-10 |
| Participants: |
| Description |
|
When determining the number of digits to round to, the existing code first uses a possible overestimate of the decimal place to round at, and corrects it if the final number of digits is not 15. However, this is incorrect when the rounding causes the decade to change: 1.00000000000000 has the same number of digits as 0.999999999999999. So, first try rounding at the smaller decimal place instead. |
| Comments |
| Comment by Githook User [ 29/Sep/16 ] |
|
Author: {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}Message: |