[SERVER-58978] Determine narrowest type before computing sum in $add in the classic engine Created: 30/Jul/21  Updated: 21/Jan/22  Resolved: 13/Jan/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.4.0, 5.0.0, 5.1.0-rc0
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Jennifer Peshansky (Inactive) Assignee: Yoon Soo Kim
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
duplicates SERVER-62485 Match the SBE $add behavior to the cl... Closed
Problem/Incident
Related
related to SERVER-61679 $sum accumulator fails in aggregation... Closed
Operating System: ALL
Steps To Reproduce:

Here is a minimal jstest that produces the inaccurate sums:

(function() {
"use strict";
 
const coll = db.sums;
coll.drop();
 
const documentList = [
    {
        _id: 100,
        "num": NumberLong("45103"),
    },
    {
        _id: 101,
        "num": NumberInt(42780),
    },
    {
        _id: 102,
        "num": NumberDecimal("100.111111111111111"),
    },
];
 
documentList.forEach(doc => assert.commandWorked(coll.insert(doc)));
 
print(JSON.stringify(
    coll.aggregate([
	    {$project: {"test": {$add: ["$num", NumberLong("9223372036854775807"), 44249]}}}
    ])
));
})();

When this test is run with SBE enabled, the printed output is:

{"_batch":[
    {"_id":102,"test":{"$numberDecimal":"9223372036854820100.111111111111111"}},
    {"_id":101,"test":9223372036854864000},
    {"_id":100,"test":9223372036854866000}
],
 "_cursorid":{"$numberLong":"0"},
 "_ns":"test.sums",
 "_db":{"_mongo":{"slaveOk":false,"host":"localhost:20000","defaultDB":"test","_defaultSession":{"_serverSession":{"handle":{}},"_isExplicit":false}},
 "_name":"test","_session":{}},
 "_collName":"sums","_cursorHandle":{}
}

This shows the final sums as
9223372036854866000,
9223372036854864000, and
9223372036854820100.111111111111111.

The real sum of these numbers is 9223372036854775807 + 44249 = 9223372036854820056, and adding the numbers in each of the documents respectively, the sums should be:
9223372036854865159,
9223372036854862836, and
9223372036854820156.111111111111111 respectively.

It appears to be rounding to (not necessarily the nearest) thousand, and, in the final case where $num is a NumberDecimal, simply adding the value of $num afterwards.

When the test is run with SBE disabled, the output is:

{"_batch":[
{"_id":102,"test":{"$numberDecimal":"9223372036854820964.111111111111111"}},
{"_id":101,"test":9223372036854864000},
{"_id":100,"test":9223372036854866000}
],
"_cursorid":{"$numberLong":"0"},"_ns":"test.sums","_db":{"_mongo":{"slaveOk":false,"host":"localhost:20000","defaultDB":"test","_defaultSession":{"_serverSession":{"handle":{}},"_isExplicit":    false}},"_name":"test","_session":{}},"_collName":"sums","_cursorHandle":{}
}

In this case, the sums for documents 100 and 101 are the same, but the sum for document 102 is:
9223372036854820964.111111111111111
which is exactly 864 higher than what the other engine had. In fact, whenever $num is a NumberDecimal, the outcomes with SBE enabled or disabled have a difference of exactly 864.

Sprint: QE 2021-08-23, QE 2021-09-06, QE 2021-09-20, QE 2021-10-04, QE 2021-10-18
Participants:
Linked BF Score: 146

 Description   

While investigating BF-21976, I discovered that both the classic and SBE engines lose precision when adding large numbers. This inaccuracy can be in the hundreds off. This bug affects at least 4.4 and 5.0, possibly earlier releases as well.

Note that this may have to do with approaching the maximum value for a NumberLong, which is NumberLong("9223372036854775807").



 Comments   
Comment by Yoon Soo Kim [ 13/Jan/22 ]

This is a dup of SERVER-62485.

Generated at Thu Feb 08 05:46:00 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.