[SERVER-14935] Long number handling Created: 18/Aug/14  Updated: 10/Dec/14  Resolved: 18/Aug/14

Status: Closed
Project: Core Server
Component/s: Shell, Storage
Affects Version/s: 2.6.3
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Tudor Alexandru Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Steps To Reproduce:

insert fairy large numbers in a collection, query for them.

Participants:

 Description   

mongos> db.user.find({session: 100000000000068636})
{ "_id" : NumberLong("1000000000066536"), "shop" : 144, "session" : [ NumberLong("100000000000068635") ] }
{ "_id" : NumberLong("1000000000066546"), "shop" : 145, "session" : [ NumberLong("100000000000068645") ] }
{ "_id" : NumberLong("1000000000066537"), "shop" : 142, "session" : [ NumberLong("100000000000068636") ], "email" : "test@gmail.com" }
{ "_id" : NumberLong("1000000000066540"), "shop" : 143, "session" : [ NumberLong("100000000000068639") ] }
{ "_id" : NumberLong("1000000000066541"), "shop" : 143, "session" : [ NumberLong("100000000000068640") ] }
{ "_id" : NumberLong("1000000000066545"), "shop" : 145, "session" : [ NumberLong("100000000000068644") ] }
{ "_id" : NumberLong("1000000000066548"), "shop" : 144, "session" : [ NumberLong("100000000000068647") ] }
{ "_id" : NumberLong("1000000000066534"), "shop" : 144, "session" : [ NumberLong("100000000000068633") ] }
{ "_id" : NumberLong("1000000000066542"), "shop" : 144, "session" : [ NumberLong("100000000000068641") ] }
{ "_id" : NumberLong("1000000000066535"), "shop" : 144, "session" : [ NumberLong("100000000000068634") ] }
{ "_id" : NumberLong("1000000000066538"), "shop" : 144, "session" : [ NumberLong("100000000000068637") ] }
{ "_id" : NumberLong("1000000000066533"), "shop" : 144, "session" : [ NumberLong("100000000000068632") ] }
{ "_id" : NumberLong("1000000000066539"), "shop" : 144, "session" : [ NumberLong("100000000000068638") ] }
{ "_id" : NumberLong("1000000000066544"), "shop" : 144, "session" : [ NumberLong("100000000000068643") ] }
{ "_id" : NumberLong("1000000000066543"), "shop" : 144, "session" : [ NumberLong("100000000000068642") ] }

versus

mongos> db.user.find({session: NumberLong("100000000000068636") })
{ "_id" : NumberLong("1000000000066537"), "shop" : 142, "session" : [ NumberLong("100000000000068636") ], "email" : "test@gmail.com" }

Although i figured out that i was calling it wrong in the first case, i can't possibly imagine what is happening behind the scenes in the first case. How can it possibly work that way? The session field isn't indexed and we run a pretty big cluster.



 Comments   
Comment by Thomas Rueckstiess [ 18/Aug/14 ]

Hi Tudor,

If a field’s value is a NumberLong (stored as 64-bit int) and you are searching using a double, the field's value will be cast to a double for comparison. This can cause precision loss on the cpp level (expected behavior). The conversion takes place in compareElementValues() in bsonelement.cpp.

Separately, the behavior of Javascript == vs === in your example is also expected. You are comparing a number primitive to a NumberLong javascript object instance. That always shortcuts to false.

Thomas

Comment by Tudor Alexandru [ 18/Aug/14 ]

Even so, it shouldn't return more than one result with the .find() function. Is the mongodb server core also running on javascript? The data was saved properly, so in my mind the query should have returned 0 results. In your own words:

mongos> number = 100000000000068636
100000000000068640
mongos> number === NumberLong("100000000000068636")
false
mongos> number === NumberLong("100000000000068635")
false
mongos> number === NumberLong("100000000000068634")
false
mongos> number === 100000000000068637
true

What is the exact internal representation of 100000000000068636 and how is it equal to the internal representation of NumberLong("xxx")..
Equality should be bijective. Inequality not.

Comment by Thomas Rueckstiess [ 18/Aug/14 ]

Hi Tudor,

MongoDB shell treats numeric values as floats by default, as described e.g. here.

For convenience, it also has internal conversion between numeric types as can be seen in this example:

// insert float
mongos> db.foo.insert({a: 1.0})
WriteResult({ "nInserted" : 1 })
 
// find as int
mongos> db.foo.find({a: 1})
{ "_id" : ObjectId("53f215ea227d2ab149267058"), "a" : 1 }

The document is returned even though the type was not the same. This includes NumberLong types as well.

// find as long
mongos> db.foo.find({a: NumberLong(1)})
{ "_id" : ObjectId("53f215ea227d2ab149267058"), "a" : 1 }

Now it's easy to see why your query would return all these documents. When you entered the long number without specifying NumberLong, the shell interpreted it at floating point precision:

mongos> number = 100000000000068636
100000000000068640
mongos> 100000000000068642 === 100000000000068636
true

All documents within the precision of the number are returned as matches.

To get the exact value back, as you discovered, you need to specify the query value as a NumberLong.

As a side note, when querying for numeric values, you can also insist on a certain type with the $type operator, see this example:

// insist on type
mongos> db.foo.find({a: 1, a: {$type: 1}})     //  1 = float
{ "_id" : ObjectId("53f215ea227d2ab149267058"), "a" : 1 }
mongos> db.foo.find({a: 1, a: {$type: 16}})    // 16 = 32bit int
mongos> db.foo.find({a: 1, a: {$type: 18}})    // 18 = 64bit int
mongos>

I hope this explains the behavior you're seeing.

I'll resolve the ticket as works as designed as I don't see an evidence for a bug here.

Regards,
Thomas

Generated at Thu Feb 08 03:36:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.