Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-75672

Classic engine incorrectly use 34 digits precision when converting from double to decimal

    • Query Execution
    • Fully Compatible
    • ALL
    • v6.0
    • QI 2023-04-17

      Classic engine use 34 digits precision when converting from double to decimal for left hand side of $add (double + decimal), while for other cases like right hand side of $add or $multiply 15 digits precision is used.

      In SBE, 15 digits precision is used in all cases.

       

      Per geert.bosch@mongodb.com, 15 digits precision is the correct one we should use

      The conversion from double to Decimal uses 15 digits by default based on research done with product management, solution architects and customers. When users add 0.02 to a Decimal amount 0.98, they expect to get an exact 1.00, not some 0.999999999999... something, even if the update was specified using a double precision number, or the other way around. Fifteen decimal digits is the largest number that can round trip through double precision without loosing precision.

            Assignee:
            davis.haupt@mongodb.com Davis Haupt (Inactive)
            Reporter:
            zixuan.zhuang@mongodb.com Zixuan Zhuang
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: