[SERVER-2814] Server does not properly store a BSON object that contains NaN Created: 22/Mar/11  Updated: 29/Aug/11  Resolved: 03/May/11

Status: Closed
Project: Core Server
Component/s: Internal Code
Affects Version/s: 1.8.0
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Ryan Nitz Assignee: Eliot Horowitz (Inactive)
Resolution: Incomplete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

1.6.6 and 1.8.0


Attachments: File mongo-java-double.tar.gz     File mongo-test.tar.gz    
Operating System: ALL
Participants:

 Description   

BSON objects that have NaN (and probably +/- infinity) are not stored in the database.



 Comments   
Comment by Ryan Nitz [ 03/May/11 ]

It's actually not a pymongo bug.

Some of the documents returned have periods in their keys.

Thanks for tracking this down Bernie!

E.g. (period is in the hostname)

"connPoolStats": {
"createdByType":

{ "master": 10276, "sync": 9 }

,
"hosts": {
"mongo-configs-1.hostname.com:26080,mongo-configs-2.hostname.com:26080,mongo-configs-3.hostname.com:26080":

{ "available": 2, "created": 9 }

,
},
"numAScopedConnection": 6,
"numDBClientConnection": 6357,
"ok": 1,
"totalAvailable": 35,
"totalCreated": 10285
},

Comment by Ryan Nitz [ 03/May/11 ]

Bernie is investigating pymongo... and the various versions of Python. Until recently, Python had weird double support.

Comment by Eliot Horowitz (Inactive) [ 03/May/11 ]

Seems to be a python issue

Comment by Ryan Nitz [ 03/May/11 ]

This seems to be working now with Java driver 2.5.3 and 1.8.1.

Attached (mongo-java-double.tar.gz) is a unit test to confirm that everything works as expected between Java and Mongo.

> db.testDouble.findOne();
{
"_id" : ObjectId("4dbf6939e51ea44031320e99"),
"nested" :

{ "MAX_VALUE" : 1.7976931348623157e+308, "MIN_VALUE" : 5e-324, "NaN" : NaN, "NEGATIVE_INFINITY" : -Infinity, "POSITIVE_INFINITY" : Infinity }

}

The BSON I'm storing via Java is being generated by Python so I'm going to see if that is the source of the problem. If that is not a problem... we can close.

Comment by Dwight Merriman [ 03/May/11 ]

the following works...suggesting not a problem? i don't usually use the java driver if you can send me something with shell that does it would be helpful.

C:\cygwin\home\dwight\mongo>mongo
MongoDB shell version: 1.9.0-pre-
connecting to: test
> y = Number.NaN
NaN
> db.aab.insert(

{z:y}

)
> db.aab.find()

{ "_id" : ObjectId("4dbf62413b544e89b091d094"), "z" : NaN }

> isNaN(3)
false
> isNaN(3.3)
false
> isNaN(db.aab.findOne().z)
true

Comment by Ryan Nitz [ 22/Mar/11 ]

Yes... I was able to hack my way around it (see previous comment).

I'll change the bug to a server bug.

Comment by Antoine Girbal [ 22/Mar/11 ]

basically the driver seems ok.
Java driver uses Double class to encode/decode the value and NaN is properly supported.
Also it can write it to JSON, but cannot parse it back (this I will fix).
Then the issue seems on server side, which doesnt like the double value when reads it.
Why do you need to use this value in MMS? Can you prevent against it?

Comment by Ryan Nitz [ 22/Mar/11 ]

As a hack for MMS, I now serialize the data to a byte array using BSON.encode (thus the byte[] is stored in Mongo) and then for display, I use BSON.decode. No exceptions are thrown.

MMS then converts the BSON to a JSON object for display in the UI (to take advantage of the org.json.JSONObject formatting options).

The NaN values display properly in the converted document (as NaN).

There is a problem with NaN in the Python driver too. Perhaps this is something that needs to be addressed in the product and drivers. I'm sure negative/positive infinity is also a problem.

Here is the hack I put in place to work around this problem:

JSONObject pingJson = null;
if (ping != null) {
final byte [] data = (byte[])ping.get("d");
if (data != null)

{ final BSONObject bson = BSON.decode(data); pingJson = new JSONObject(bson.toString()); }

else

{ pingJson = new JSONObject(); }

} else

{ pingJson = new JSONObject(); }

params.put("serverResponse", pingJson.toString(4));

Comment by Antoine Girbal [ 22/Mar/11 ]

looks like there is bad data in the record, some double values are "NaN".
Maybe some kind of division by 0?
When you print json you can see "avgObjSize" : NaN
I'm guessing the bson decoder on server side doesnt like it..

Comment by Antoine Girbal [ 22/Mar/11 ]

If you set a safe writeconcern, you get this exception, so server doesnt like record.
1st going to check that obj looks ok on the client side

$ ant run
Buildfile: /home/antoine/Downloads/mongo-test/build.xml

compile:
[javac] Compiling 1 source file to /home/antoine/Downloads/mongo-test/build/java/classes

run:
[junit] Testsuite: com.deftlabs.tests.mongo.RandomTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.481 sec
[junit]
[junit] Testcase: testBsonUpsert(com.deftlabs.tests.mongo.RandomTest): Caused an ERROR
[junit] not okForStorage
[junit] com.mongodb.MongoException: not okForStorage
[junit] at com.mongodb.DBTCPConnector._checkWriteError(DBTCPConnector.java:141)
[junit] at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:162)
[junit] at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:146)
[junit] at com.mongodb.DBApiLayer$MyCollection.update(DBApiLayer.java:335)
[junit] at com.mongodb.DBCollection.update(DBCollection.java:145)
[junit] at com.deftlabs.tests.mongo.RandomTest.testBsonUpsert(RandomTest.java:58)
[junit]
[junit]

Comment by Antoine Girbal [ 22/Mar/11 ]

k indeed didnt see any data but I assumes the collection got deleted during test cleanup.
looking into it now

Comment by Ryan Nitz [ 22/Mar/11 ]

You see the data in the collection (test.bsonSave)? I ran it on 2.5/1.8 and it didn't insert the data. It didn't throw an exception, but there wasn't any data in the db.

Comment by Antoine Girbal [ 22/Mar/11 ]

does it fail always?
so far I ran it 10 times with 2.5 driver and mongo 1.8 and no failure.
What kind of failure / exception are you seeing?

Generated at Thu Feb 08 03:01:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.