[SERVER-6618] constant folding implementation assumes $add can can be decomposed (is "commutative and associative") but typed addition with overflow cannot Created: 27/Jul/12  Updated: 06/Dec/22  Resolved: 04/Aug/20

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Aaron Staple Assignee: Backlog - Query Team (Inactive)
Resolution: Done Votes: 0
Labels: query-44-grooming
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query
Operating System: ALL
Participants:

 Description   

Observed behavior: The results of some additions may depend on whether the values come from a constant or a non constant expression.
Expected behavior: The result of an addition depends on values only.

Test:

c = db.c;
c.drop();
 
c.save( { a:0.0 } );
 
large = NumberLong('9223372036854775807');
 
// 1. Two large longs overflow.
printjson( c.aggregate( { $project:{ sum:{ $add:[ large, large ] } } } ) );
// 2. Two large longs are cast to doubles if they are summed with a double, and do not overflow.
printjson( c.aggregate( { $project:{ sum:{ $add:[ large, large, 0.0 ] } } } ) );
// 3. The constant folding implementation sums the first two operands, which overflow, then sums the overflow result with the value of '$a'.
printjson( c.aggregate( { $project:{ sum:{ $add:[ large, large, '$a' ] } } } ) );

Results:

// 1
{
	"result" : [
		{
			"_id" : ObjectId("50121f8b8e4c36714e472e38"),
			"sum" : NumberLong(-2)
		}
	],
	"ok" : 1
}
// 2
{
	"result" : [
		{
			"_id" : ObjectId("50121f8b8e4c36714e472e38"),
			"sum" : 18446744073709552000
		}
	],
	"ok" : 1
}
// 3
{
	"result" : [
		{
			"_id" : ObjectId("50121f8b8e4c36714e472e38"),
			"sum" : -2
		}
	],
	"ok" : 1
}



 Comments   
Comment by Craig Homa [ 04/Aug/20 ]

This work was done somewhere along the way in the last few years, and as per Asya's comment, now works in 4.4.0.

Comment by Asya Kamsky [ 03/Aug/20 ]

In latest (4.4.0) the behavior is now consistent across all three examples.

All three give the same result now:

printjsononeline( c.aggregate( { $project:{ sum:{ $add:[ large, large ] } } } ).toArray()[0] );
{  "_id" : ObjectId("5f284ab498826faa09f58f28"),  "sum" : 18446744073709552000 }
printjsononeline( c.aggregate( { $project:{ sum:{ $add:[ large, large, 0.0 ] } } } ).toArray()[0] );
{  "_id" : ObjectId("5f284ab498826faa09f58f28"),  "sum" : 18446744073709552000 }
printjsononeline( c.aggregate( { $project:{ sum:{ $add:[ large, large, "$a" ] } } } ).toArray()[0] );
{  "_id" : ObjectId("5f284ab498826faa09f58f28"),  "sum" : 18446744073709552000 }

This looks like we did the second option, as opposed to the first that was discussed as preferred. I don't think we can go back so we should close this as gone away or duplicate of the ticket that actually fixed this in some version.

Comment by Charlie Swanson [ 07/Dec/15 ]

After discussing with redbeard0531, we've decided that we should do the following:

  • Implement some method to manually coerce one type to another (see SERVER-5239).
  • Error when we would overflow in $add and $sum.

This would make it so that the first case and the third case would fail with an error, which you could avoid if you needed to by explicitly converting the longs to doubles. The error message for overflow should probably mention that you can up-convert explicitly.

An alternative would be to automatically convert overflows of longs into a double. The first option was preferred because it would allow you to do that if you wanted, but would make it obvious when that is happening.

Comment by Aaron Staple [ 27/Jul/12 ]

Tentatively assigning to 2.3.0

Generated at Thu Feb 08 03:12:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.