[SERVER-5209] can't compare values of BSON types 16 and 18 Created: 05/Mar/12  Updated: 05/Nov/15  Resolved: 07/Mar/12

Status: Closed
Project: Core Server
Component/s: Aggregation Framework, Internal Code
Affects Version/s: 2.1.0
Fix Version/s: 2.1.1

Type: Bug Priority: Major - P3
Reporter: Tim Garvin Assignee: Chris Westin
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Debian 5, Kernel 2.6.26, 32-bit OS.


Attachments: Text File aggregationGroupBy.txt    
Issue Links:
Related
related to SERVER-5239 $project should have numeric type coe... Closed
Operating System: Linux
Participants:

 Description   

When testing the aggregation framework, tried to group by a field that has been previously stored as both 64-bit and 32-bit numbers, depending on the number.

Running the aggregation query results in a code 16016, or "exception: can't compare values of BSON types 16 and 18"

Running the old db.collection.group query still works as intended.

Heavily simplified versions of the aggregation framework query (not working) and the original collection.group query (working) are attached. In both of these queries, it is understood that the field 'ip1' may contain either a 32-bit number (BSON type 16) or a 64-bit number (type 18) I would expect both queries to return groupings based on the field 'ip1'.

A probable work-around (implementing this currently) is to somehow ensure that all data is written as a 64-bit number, so that all comparisons are between like types, but this seems like something that a green mongo user might find cumbersome to have to understand and deal with.

New to the bug-fix cycle, and to mongodb in general-- let me know if there's more information I can provide.



 Comments   
Comment by Chris Westin [ 07/Mar/12 ]

$sort may also be affected, since the numeric comparisons have been adjusted to work across numeric data types.

Comment by Sam Kleinman (Inactive) [ 07/Mar/12 ]

does this only affect the $group operator, or are other pipeline operators similarly impacted?

Comment by Chris Westin [ 07/Mar/12 ]

BSON allows the use of several numeric types, not just JavaScript's
double precision floating point numbers. We notate these special numeric
types as demonstrated by SERVER-5234.

Aggregation (as well as any other operations that operate over an entire
collection) works best if the data types of the fields are homogeneous. The
$group code assumes that field types are homogeneous (this was previously
undocumented), and that led to this bug: the user changed the way they stored
this field mid-way through lifetime of this collection. Because of the above
assumptions, the $group code didn't group together (for example) the values
NumberInt(24), NumberLong(24), and 24 (the JavaScript double precision floating
point number).

The fix adds code so that those values are all treated as being the same.
However, there are some internal details that will still show through. When
a new group bucket is created, the first value for that bucket will be used
as its key. If a collection uses a mixed set of types for a value that is
used for a grouping key, then the output will be similarly mixed, because the
order that values of these types are seen for different values may not
necessarily be the same. Clients should be aware of this difference, which
will manifest in strongly typed programming languages such as C/C++ or Java.
This shouldn't introduce any additional complexity, because if the application
was using mixed types before, then it already must have logic for detecting
the types of these values after they are fetched. This is simply a warning
that this requirement will continue into their use of aggregation results.

Comment by Sam Kleinman (Inactive) [ 07/Mar/12 ]

Could you illuminate the documentation change that you think is required? It seems like the previous behavior wasn't documented, but perhaps I'm missing something.

Comment by auto [ 07/Mar/12 ]

Author:

{u'login': u'cwestin', u'name': u'U-tellus\\cwestin', u'email': u'cwestin@10gen.com'}

Message: SERVER-5209 aggregation promotes numeric values when necessary
Branch: master
https://github.com/mongodb/mongo/commit/05604645b421fa934b2db313b0d96a0bcc4fb5ad

Generated at Thu Feb 08 03:08:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.