Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-58978

Determine narrowest type before computing sum in $add in the classic engine

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: 4.4.0, 5.0.0, 5.1.0-rc0
    • Fix Version/s: Backlog
    • Component/s: None
    • Labels:
      None
    • Operating System:
      ALL
    • Steps To Reproduce:
      Hide

      Here is a minimal jstest that produces the inaccurate sums:

      (function() {
      "use strict";
       
      const coll = db.sums;
      coll.drop();
       
      const documentList = [
          {
              _id: 100,
              "num": NumberLong("45103"),
          },
          {
              _id: 101,
              "num": NumberInt(42780),
          },
          {
              _id: 102,
              "num": NumberDecimal("100.111111111111111"),
          },
      ];
       
      documentList.forEach(doc => assert.commandWorked(coll.insert(doc)));
       
      print(JSON.stringify(
          coll.aggregate([
      	    {$project: {"test": {$add: ["$num", NumberLong("9223372036854775807"), 44249]}}}
          ])
      ));
      })();
      

      When this test is run with SBE enabled, the printed output is:

      {"_batch":[
          {"_id":102,"test":{"$numberDecimal":"9223372036854820100.111111111111111"}},
          {"_id":101,"test":9223372036854864000},
          {"_id":100,"test":9223372036854866000}
      ],
       "_cursorid":{"$numberLong":"0"},
       "_ns":"test.sums",
       "_db":{"_mongo":{"slaveOk":false,"host":"localhost:20000","defaultDB":"test","_defaultSession":{"_serverSession":{"handle":{}},"_isExplicit":false}},
       "_name":"test","_session":{}},
       "_collName":"sums","_cursorHandle":{}
      }
      

      This shows the final sums as
      9223372036854866000,
      9223372036854864000, and
      9223372036854820100.111111111111111.

      The real sum of these numbers is 9223372036854775807 + 44249 = 9223372036854820056, and adding the numbers in each of the documents respectively, the sums should be:
      9223372036854865159,
      9223372036854862836, and
      9223372036854820156.111111111111111 respectively.

      It appears to be rounding to (not necessarily the nearest) thousand, and, in the final case where $num is a NumberDecimal, simply adding the value of $num afterwards.

      When the test is run with SBE disabled, the output is:

      {"_batch":[
      {"_id":102,"test":{"$numberDecimal":"9223372036854820964.111111111111111"}},
      {"_id":101,"test":9223372036854864000},
      {"_id":100,"test":9223372036854866000}
      ],
      "_cursorid":{"$numberLong":"0"},"_ns":"test.sums","_db":{"_mongo":{"slaveOk":false,"host":"localhost:20000","defaultDB":"test","_defaultSession":{"_serverSession":{"handle":{}},"_isExplicit":    false}},"_name":"test","_session":{}},"_collName":"sums","_cursorHandle":{}
      }
      

      In this case, the sums for documents 100 and 101 are the same, but the sum for document 102 is:
      9223372036854820964.111111111111111
      which is exactly 864 higher than what the other engine had. In fact, whenever $num is a NumberDecimal, the outcomes with SBE enabled or disabled have a difference of exactly 864.

      Show
      Here is a minimal jstest that produces the inaccurate sums: (function() { "use strict" ;   const coll = db.sums; coll.drop();   const documentList = [ { _id: 100 , "num" : NumberLong( "45103" ), }, { _id: 101 , "num" : NumberInt( 42780 ), }, { _id: 102 , "num" : NumberDecimal( "100.111111111111111" ), }, ];   documentList.forEach(doc => assert .commandWorked(coll.insert(doc)));   print(JSON.stringify( coll.aggregate([ {$project: { "test" : {$add: [ "$num" , NumberLong( "9223372036854775807" ), 44249 ]}}} ]) )); })(); When this test is run with SBE enabled, the printed output is: { "_batch" :[ { "_id" : 102 , "test" :{ "$numberDecimal" : "9223372036854820100.111111111111111" }}, { "_id" : 101 , "test" : 9223372036854864000 }, { "_id" : 100 , "test" : 9223372036854866000 } ], "_cursorid" :{ "$numberLong" : "0" }, "_ns" : "test.sums" , "_db" :{ "_mongo" :{ "slaveOk" : false , "host" : "localhost:20000" , "defaultDB" : "test" , "_defaultSession" :{ "_serverSession" :{ "handle" :{}}, "_isExplicit" : false }}, "_name" : "test" , "_session" :{}}, "_collName" : "sums" , "_cursorHandle" :{} } This shows the final sums as 9223372036854 866000 , 9223372036854 864000 , and 9223372036854 820100 .111111111111111. The real sum of these numbers is 9223372036854 775807 + 44249 = 9223372036854 820056 , and adding the numbers in each of the documents respectively, the sums should be: 9223372036854 865159 , 9223372036854 862836 , and 9223372036854 820156 .111111111111111 respectively. It appears to be rounding to (not necessarily the nearest) thousand, and, in the final case where $num is a NumberDecimal, simply adding the value of $num afterwards. When the test is run with SBE disabled, the output is: { "_batch" :[ { "_id" : 102 , "test" :{ "$numberDecimal" : "9223372036854820964.111111111111111" }}, { "_id" : 101 , "test" : 9223372036854864000 }, { "_id" : 100 , "test" : 9223372036854866000 } ], "_cursorid" :{ "$numberLong" : "0" }, "_ns" : "test.sums" , "_db" :{ "_mongo" :{ "slaveOk" : false , "host" : "localhost:20000" , "defaultDB" : "test" , "_defaultSession" :{ "_serverSession" :{ "handle" :{}}, "_isExplicit" : false }}, "_name" : "test" , "_session" :{}}, "_collName" : "sums" , "_cursorHandle" :{} } In this case, the sums for documents 100 and 101 are the same, but the sum for document 102 is: 9223372036854 820964 .111111111111111 which is exactly 864 higher than what the other engine had. In fact, whenever $num is a NumberDecimal, the outcomes with SBE enabled or disabled have a difference of exactly 864.
    • Sprint:
      QE 2021-08-23, QE 2021-09-06, QE 2021-09-20, QE 2021-10-04, QE 2021-10-18
    • Linked BF Score:
      35

      Description

      While investigating BF-21976, I discovered that both the classic and SBE engines lose precision when adding large numbers. This inaccuracy can be in the hundreds off. This bug affects at least 4.4 and 5.0, possibly earlier releases as well.

      Note that this may have to do with approaching the maximum value for a NumberLong, which is NumberLong("9223372036854775807").

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              backlog-query-execution Backlog - Query Execution
              Reporter:
              jennifer.peshansky Jennifer Peshansky
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

                Dates

                Created:
                Updated: