[JAVA-4073] ObjectId not deserializing from json to its original value Created: 17/Mar/21  Updated: 27/Oct/23  Resolved: 18/Mar/21

Status: Closed
Project: Java Driver
Component/s: BSON, POJO
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Subash Lingaredd Assignee: Ross Lawley
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

I have a rest api which reads a document from mongo collection and one more API which adds/updates the document into the same collection.

 

I have upgraded project to Spring-data-mongo 2.4.3 and now I have a problem.

 

Below is Objectid for a row

 

57d7171c02666d68cb2ec50b

It translates to the corresponding json

{ "timestamp": 1473713948, "counter": 3065099, "randomValue1": 157293, "randomValue2": 26827 }

and in API response I get below output

{{ "id":

{ "timestamp": 1473713948, "counter": 3065099, "randomValue1": 157293, "randomValue2": 26827 }

,
"product": "ABCD"}

 

Now if I update the product name to something else and use the second API to post the same content, instead of updating the row, it is creating a different row.

*{{ "id":

{ "timestamp": 1473713948, "counter": 3065099, "randomValue1": 157293, "randomValue2": 26827 }

,*
"product": "EFGH"}

 

Now the new ObjectId for new row is below

57d7171c07430961eb07d9e2

 

Intrestingl both translates to the same json as above.**

 

this is happening in latest version of spring-data-mongo only.... earlier version 2.0.1 is good.****

Not sure if the issue is with BSON ID, or JACKSON or MONGO DRIVER itself.****

 

Please assist us on this issue.****

 

 



 Comments   
Comment by Ross Lawley [ 18/Mar/21 ]

Hi lingared@adobe.com,

Just to note we don't support Spring-data-mongo or Jackson. However, I have found a stackoverflow post that provides an example of ObjectId serialization with Jackson.

I'm still unclear if the API that provides the JSON uses the same version of the Spring data as the API that consumes the JSON.

I would advise you to use our MongoDB community portal, located here for this question. There maybe users that have had similar issues.

All the best,

Ross

 

Comment by Subash Lingaredd [ 17/Mar/21 ]

Could u please provide us some working java example for below 

 

If you can configure Jackson to use the ObjectId.toHexString() method and ObjectId(final String hexString) constructor, then that would be a better approach as it relies on the public API.

 

I checked a few but no success.**

 

Still I am in a state of confusion whether this is a mongo-java-driver version change issue or a Jackson issue because of this version change.****

Comment by Ross Lawley [ 17/Mar/21 ]

Hi lingared@adobe.com,

The ObjectId class was brought up to specification in JAVA-2916. This is when the change happened to the internal implementation details of the ObjectId class. It looks like the way in which you are using Jackson is causing these internal private fields to be serialized. This is leaking the internal implementation details of the ObjectId class! Which generally should be avoided as it risks compatibility issues like you describe.

We suspect this is happening due to a change in machineIdentfier and processIdentifier fields. All the old rows are serialized with these fields using 3.6.3 and now we are deserializing with 4.1.1 we suspect the deserializer is not able to find those machineIdentifier and processIdentifier fields and instead it finds randomValue1 and randomValue2 fields and hence it declares it a new id and inserts it.

This shouldn't be an issue if both the provider and consumer of the Json use the same version of the Java driver underneath and a serialization mechanism that supports the 4.1.1 version of the ObjectId. I'm not sure how Jackson deserializes into ObjectId classes. It should be noted that there is no constructor that takes ObjectId(final int timestamp, final int randomValue1, final short randomValue2, final int counter) so using those values is not recommended. I'm not sure how Jackson deserializes the json: {"timestamp": 1473713948, "counter": 3065099, "randomValue1": 157293, "randomValue2": 26827} into an ObjectId -  so that should be investigated.

If you can configure Jackson to use the ObjectId.toHexString() method and ObjectId(final String hexString) constructor, then that would be a better approach as it relies on the public API.

Ross

Comment by Subash Lingaredd [ 17/Mar/21 ]

We use JACKSON for serializing and deserializing.

Comment by Subash Lingaredd [ 17/Mar/21 ]

Hi Ross,

thanks for a quick response.

Let me explain the chronology

Earlier we were using Spring Data Mongo 2.0.0.RELEASE and this has mongo-driver 3.6.3 installed. Everything was working fine and also in the above-mentioned scenario, the update operation will update the same row again and again (if it has an id field, the id field will do as our spring manages both id and _id)

Now we are planning to move to Spring Data Mongo 2.4.3 and this has mongo-driver 4.1.1 installed with it. Here comes the problem, the same scenario explained above is now inserting n number of rows (for n number of updates on the same row with id field). Even the new rows are also behaving the same way.

We suspect this is happening due to a change in machineIdentfier and processIdentifier fields. All the old rows are serialized with these fields using 3.6.3 and now we are deserializing with 4.1.1  we suspect the deserializer is not able to find those machineIdentifier and processIdentifier fields and instead it finds randomValue1 and randomValue2 fields and hence it declares it a new id and inserts it.

 

We have tested it 3 to 4 times today with these two mongo-driver version and was able to replicate the issue.

the weird thing is the same happens again if I update the new row (which was created using 4.1.1) again.

Comment by Ross Lawley [ 17/Mar/21 ]

Hi lingared@adobe.com,

JAVA-3854 relates to JVM Object serialization and deserialization of the ObjectId class via writeReplace and readResolve.

The issue you describe seems to be related to JSON serialization and deserialization. Its unclear what mechanism you are using to serialize the ObjectId value (Jackson?). I would recommend using extended JSON for serializing bson documents. This is more robust as it doesn't rely on private internal implementation details of the class being serialized. That said both seem to produce the same data from the example given.

Just to note the java driver will add a _id field for a document if one is missing. I notice that you are passing id fields and not _id fields - could that be the issue? And the reason you are getting two inserts?

Without more information regarding how you are using the driver and creating the values for the API - its difficult to understand what the actual issue is. Ideally, a minimal reproducible example would help as I could replicate the issue.

Please use one of the support mechanisms listed above and support can be given there. Feel free to post a link to this ticket and I can facilitate escalating your issue.

All the best,

Ross

Comment by Subash Lingaredd [ 17/Mar/21 ]

https://jira.mongodb.org/browse/JAVA-3854

 

I see a link to this. Please let us know how should we be backward compatible....

Comment by Ross Lawley [ 17/Mar/21 ]

Hi lingared@adobe.com,

Thank you for reaching out. As this sounds like a support issue, I wanted to give you some resources to get this question answered more quickly:

  • Our MongoDB support portal, located at support.mongodb.com
  • Our MongoDB community portal, located here
  • If you are an Atlas customer, there is free support offered 24/7 in the lower right hand corner of the UI

Just in case you have already opened a support case and are not receiving sufficient help, please let me know and I can facilitate escalating your issue.

All the best,

Ross Lawley

Comment by Subash Lingaredd [ 17/Mar/21 ]

Earlier our json was like below

 

{             "timestamp": 1473713948,             "counter": 3065099,             "machineIdentifier": 157293,             "processIdentifier": 26827         }

Are issues because of change in field names for machineIdentifier and processIdentifier in new version?

Comment by Subash Lingaredd [ 17/Mar/21 ]

Class we use for Mongo is MongoOperations

 

Comment by Subash Lingaredd [ 17/Mar/21 ]

I am using SAVE method which inserts if no id is present and updates if there is an existing id field.

Generated at Thu Feb 08 09:01:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.