Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-26193

conversion from double to decimal rounds 0.999999999999999 incorrectly

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • 3.4.0-rc0
    • None
    • Shell
    • None
    • Fully Compatible
    • ALL
    • Hide

      > x=NumberDecimal(0.999999999999999)
      NumberDecimal("1.00000000000000")

      Show
      > x=NumberDecimal(0.999999999999999) NumberDecimal("1.00000000000000")
    • Integration 2016-10-10

    Description

      When determining the number of digits to round to, the existing code first uses a possible overestimate of the decimal place to round at, and corrects it if the final number of digits is not 15. However, this is incorrect when the rounding causes the decade to change: 1.00000000000000 has the same number of digits as 0.999999999999999. So, first try rounding at the smaller decimal place instead.

      Attachments

        Activity

          People

            geert.bosch@mongodb.com Geert Bosch
            geert.bosch@mongodb.com Geert Bosch
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: