[JAVA-2702] BasicDBObject.parse fails to parse json if it contains big numbers Created: 13/Dec/17  Updated: 27/Oct/23  Resolved: 14/Dec/17

Status: Closed
Project: Java Driver
Component/s: JSON
Affects Version/s: 3.4.3, 3.6.0
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Mikhail Danilov Assignee: Ross Lawley
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Input:
JSON with big number (exceeds java.lang.Long limits).
Sample code:

BasicDBObject obj = BasicDBObject.parse("{\"bigNum\": 11111111111111111111111}");
System.out.println(obj);

Expected behavior:
BasicDBObject.parse handles such number as Decimal128 (added in 3.4)

Actual behavior:
BasicDBObject.parse fails with NumberFormatException. (JsonScanner.scanNumber tries to parse number as Integer, Long or Double, but not Decimal128).

Exception in thread "main" java.lang.NumberFormatException: For input string: "11111111111111111111111"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Long.parseLong(Long.java:592)
	at java.lang.Long.parseLong(Long.java:631)
	at org.bson.json.JsonScanner.scanNumber(JsonScanner.java:437)
	at org.bson.json.JsonScanner.nextToken(JsonScanner.java:93)
	at org.bson.json.JsonReader.popToken(JsonReader.java:492)
	at org.bson.json.JsonReader.readBsonType(JsonReader.java:136)
	at com.mongodb.DBObjectCodec.readDocument(DBObjectCodec.java:345)
	at com.mongodb.DBObjectCodec.decode(DBObjectCodec.java:138)
	at com.mongodb.DBObjectCodec.decode(DBObjectCodec.java:61)
	at com.mongodb.BasicDBObject.parse(BasicDBObject.java:74)
	at com.mongodb.BasicDBObject.parse(BasicDBObject.java:61)



 Comments   
Comment by Jeffrey Yemin [ 14/Dec/17 ]

mdanilov thanks for the feedback and for working through this issue. For now, your best bet is to pre-process your JSON, as you would have to do with other MongoDB BSON types that have no native JSON representation (e.g. ObjectId, Date)

Comment by Mikhail Danilov [ 14/Dec/17 ]

Jeff, thanks for explanation.
I suggest to use Decimal128 only where Double or Long are not applicable (currently using size as the only defining factor) as an alternative to throwing NumberFormatException.
It will provide more use cases for JsonReader-based parsers and hopefully will not be confusing for clients. (In my case js-style json strings are provided by 3rd party and it seems like I'm forced to do some processing to make it valid extended json before feeding to mongo parsers to overcome NumberFormatException).

Comment by Jeffrey Yemin [ 14/Dec/17 ]

JsonReader already supports Decimal128 but requires a specific escape syntax to specify values of that type, as specified in our Extended JSON specification. The JSON input has to look like this:

   {"d" : {"$numberDecimal" : "1.400"}}

The rationale for distinguishing Decimal128 values from Double values is that they don't just differ in size, but in semantics. Decimal128 values can represent decimal values like 0.1 exactly, for example, while Double values can not. They can also represent trailing zeros losslessly, so, for example, 1.40 is distinct from 1.400. See the Wikipedia article for more details.

Comment by Mikhail Danilov [ 14/Dec/17 ]

BasicDBObject#parse uses JsonReader under the hood. And JsonReader's javadoc says:

Reads a JSON in one of the following modes:
Strict mode that conforms to the JSON RFC specifications.
JavaScript mode that that most JavaScript interpreters can process
Shell mode that the mongo shell can process. This is also called "extended" JavaScript format.

Also in driver documentation http://mongodb.github.io/mongo-java-driver/3.6/bson/extended-json/:

JsonReader automatically detects the JSON flavor in the string, so you do not need to specify it.

Based on that I guess that feature request for Decimal128 support in JsonReader/JsonScanner should be created. And this issue should be discarded since BasicDBObject#parse correctly handles json according to javadoc.

Any thoughts on that?

Comment by Mikhail Danilov [ 13/Dec/17 ]

Hello @ross.lawley,

Thanks for quick response.

I used BasicDBObject as a replacement for com.mongodb.util.JSON (is deprecated in 3.6) and extended-json was not mentioned in JSON's javadoc.

Does driver offers any other tools to parse arbitrarary json string (non-extended) to BasicDBObject?

Comment by Ross Lawley [ 13/Dec/17 ]

Hi mdanilov,

Thanks for the ticket, this is currently expected behaviour as BasicDBObject#parse handles extended-json, as stated in the API documentation:

Parses a string in MongoDB Extended JSON format to a BasicDBObject

Ross

Generated at Thu Feb 08 08:57:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.