[JAVA-2483] Add support for Jackson Java JSON serialization library Created: 04/Apr/17  Updated: 18/Jan/24

Status: Backlog
Project: Java Driver
Component/s: JSON
Affects Version/s: None
Fix Version/s: None

Type: Epic Priority: Major - P3
Reporter: Ralph Jennings Assignee: Unassigned
Resolution: Unresolved Votes: 8
Labels: rp-track
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by JAVA-1314 Create Jackson-based implementation o... Closed
Scope Cost Estimate: 0
Cost to Date: 0
Final Cost Estimate: 0
Cost Threshold %: 100

 Description   

Document.toJson() produces jackson-incompatible mongo extended JSON.

The JSON.serialize method (which produces jackson-compatible JSON) has been deprecated.

Since Jackson is THE standard Java JSON serialization library... If you are writing such a library for Java, then you should consider making Jackson support a priority.

Ideally, this would allow us to use a jackson based library to get a MongoCollection<MyJacksonSerializableObject> directly from the mongo drivers.

The current workaround of getting a MongoCollection<Document>, turning it into Jackson-compatible JSON (via the @deprecated JSON.serialize) and turning that into MyJacksonSerializableObject via an ObjectMapper, though less than ideal, is at least a workable solution. So if the ideal solution is not possible, then please at least include a standard Jackson-compatible JSON serializer.

Some research to support this feature:

 



 Comments   
Comment by Vishal Surana [ 13/Apr/20 ]

Are there any plans to support Jackson serialization/deserialization? Bson annotations are nowhere near as powerful and expressive as Jackson and it may not be practical to implement and update codecs for any non-trivial changes to the domain model. 

There are libraries such as bson4jackson, MongoJack, Jongo, etc. which attempt to solve this problem but they have their own set of issues and aren't updated as frequently as official drivers.

Comment by Jeffrey Yemin [ 28/Nov/18 ]

james.broadhead can you elaborate more on this?  We're removing the JSON class because it implements an unspecified and non-standard "extended" JSON format.  The replacement for that is JsonReader and JsonWriter, which are implemented to the specification.  My understanding is that Jackson-compatibile serialization is more about supporting Jackson annotations for controlling the conversion to and from BSON. 

Comment by James Broadhead (Inactive) [ 28/Nov/18 ]

As a note - Cloud is a heavy user of JSON.serialize (via our JsonUtils wrapper - https://github.com/10gen/mms/blob/master/server/src/main/com/xgen/svc/core/util/json/JsonUtils.java#L271).

If you're planning on removing the JSON class, having supported Jackson-compatible serialization first would be a big plus

Comment by Jeffrey Yemin [ 25/Jan/18 ]

A community codec is available at https://github.com/ylemoigne/mongo-jackson-codec.

Comment by Ralph Jennings [ 21/Apr/17 ]

One other thing to consider...

All of this came about because the following assertion fails:

Assert.assertEquals(jsonText, Document.parse(jsonText).toJson());

The issue isn't that we are losing information from BSON, but that the java drivers are adding invalid information when creating the BSON.

If I provide some JSON text to Document.parse, I expect document.toJson() to produce identical output (ignoring whitespace).

When I try this locally using the following JSON text, everything but "myLong" appears to come back correctly:

{
  "myNull" : null,
  "myString" : "A string",
  "myBoolean" : false,
  "myInt" : 214748364,
  "myLong" : 9223372036854775807,
  "myDouble" : 1.7976931348623157E308,
  "myDate" :      "2016-12-02T16:47:04.192Z",
  "myDecimal" : 1.1,
  "myObjectId" : "58e457fd5ac0fa50bd537a17"
}

Comment by Ralph Jennings [ 21/Apr/17 ]

Ideally, the java mongo drivers would just convert the BSON straight to a POJO using the jackson annotations (or jackson style reflection).

I never ran across the issues you mention with "myDate", "myDecimal", "myObjectId", or "myBinary" when using the deprecated JSON.serialize method.

I don't use Date objects, however there seems to be 2 standard ways of representing them, either as a long value (number of millis past the epoch – such as from Date.getTime()), or as a text value (ISO 8601 formatted string). I believe jackson will work with either of those representations out-of-the-box.

All of my numbers looked as I expected (as shown in your "myInt", "myLong", and "myDouble" examples).

I stored my id as a String (using ObjectId.get().toHexString()) internally, but it seems clear how you would do such a conversion if the member variable was of type ObjectId instead. In jackson, for special object types, you can inject de/serialization modules to the ObjectMapper, or you can annotate any member variable with de/serialization module specifics:

@JsonSerialize(using = MySerializer.class) // takes objectId and returns objectId.toHexString();
@JsonDeserialize(using = MyDeserializer.class) // takes hexString and returns new ObjectId(hexString);
private ObjectId objectId;

I never handled binary data directly, but I suppose if your java POJO had a "myBinary" variable, it would probably be a byte[], and the conversion should be obvious.

From your examples, it seems like there would be some difficulties (wrt ObjectId or binary data) in creating the non-ideal-solution universal jackson serializer. However, if the library instead did the ideal solution of converting between BSON and POJO internally, there seem to be obvious ways to handle those types. If there aren't, then I see no problem with throwing some sort of Exception (or even a RuntimeException)) when there is a failure in conversion (somebody tried to stick an ObjectId into an int variable, or an Integer value into a MyPojo object for instance, or set the value of a final field, etc) as jackson will do the same thing if it finds something it didn't expect.

Comment by Jeffrey Yemin [ 05/Apr/17 ]

Thanks for opening this issue.

In order to proceed, we would need to determine what Jackson-compatible JSON even means for every BSON type supported by the driver. Here's some sample output from JSON#serialize for the most common BSON types:

{ 
  "myNull" : null,
  "myString" : "A string , 
  "myBoolean" : false, 
  "myInt" : 214748364 , 
  "myLong" : 9223372036854775807, 
  "myDouble" : 1.7976931348623157E308,
  "myDate" :      { "$date" : "2016-12-02T16:47:04.192Z}", 
  "myDecimal" : { "$numberDecimal" : "1.1"},
  "myObjectId" : { "$oid" : "58e457fd5ac0fa50bd537a17"},
  "myBinary" : <Binary Data> 
}

The last four (date, binary, decimal128, ObjectId) would be problematic for a default-configured Jackson ObjectMapper. The value for myBinary isn't even valid JSON (though JSONSerializers.getStrict()#serialize produces something more reasonable for binary).

One option that will be available to driver users in the next release is the ability to take full control of the JSON produced by any of the toJson method by registering custom converters. For example:

        JsonWriterSettings settings = JsonWriterSettings.builder()
                                              .int64Converter((value, writer) -> writer.writeNumber(value.toString()))
                                              .dateTimeConverter((value, writer) -> writer.writeString(new SimpleDateFormat("yyyy-MM-dd'T'HH:mm'Z'").format(new Date(value))))
                                              .build();
        Document doc = new Document("i", 12)
                               .append("l", Long.MAX_VALUE)
                               .append("d", new Date());
 
        String json = doc.toJson(settings);

will produce JSON like this:

{ "i" : 12, "l" : 9223372036854775807, "d" : "2017-04-19T09:12Z" }

as opposed to the default representation:

{ "i" : 12, "l" : { "$numberLong" : "9223372036854775807" }, "d" : { "$date" : 1492607615102 } }

So perhaps a way forward would be to provide Jackson-compatible converters for the BSON types that Jackson supports, but not to provide a new mode that can only support a subset of BSON types.

Generated at Thu Feb 08 08:57:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.