[JAVA-5099] Driver is using Kotlin native serializer over custom Codec. Created: 07/Aug/23 Updated: 27/Oct/23 Resolved: 09/Aug/23 |
|
| Status: | Closed |
| Project: | Java Driver |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Question | Priority: | Major - P3 |
| Reporter: | Uroš Jarc | Assignee: | Ross Lawley |
| Resolution: | Works as Designed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||
| Issue Links: |
|
||||||||||||||||
| Documentation Changes Summary: | 1. What would you like to communicate to the user about this feature? |
||||||||||||||||
| Description |
| Comments |
| Comment by Uroš Jarc [ 09/Aug/23 ] | ||||||||||||||||||||||||||||||||||||||
|
Here is finaly managed to write it: https://www.mongodb.com/community/forums/t/confusion-how-kotlinx-and-codec-serialization-flow-is-working-in-kotlin-java-mongodb-driver/238929 | ||||||||||||||||||||||||||||||||||||||
| Comment by Ross Lawley [ 09/Aug/23 ] | ||||||||||||||||||||||||||||||||||||||
|
I'll wait for the link before closing. | ||||||||||||||||||||||||||||||||||||||
| Comment by Uroš Jarc [ 09/Aug/23 ] | ||||||||||||||||||||||||||||||||||||||
|
I see! Ok lets do it like this... This issue is much more complex and not so clear as it seems, you have to really know what is happening in the background to do it right... Let's close this issue as you suggested and I will provide a link to the post on the community forum because I have only one more question... | ||||||||||||||||||||||||||||||||||||||
| Comment by Ross Lawley [ 09/Aug/23 ] | ||||||||||||||||||||||||||||||||||||||
I've added JAVA-5103 as it shouldnt NPE but the issue is the Codec is not properly formed. Please see the: Codecs documentation MongoDB stores data in Bson Documents, each value has a key (String) and a Bson Value. When storing data or reading data from MongoDB, Codecs are responsible for translating that data into a valid Bson Document}}s. The {{ImageCodec doesn't do this it just translates a single value and as such is expected to error (albeit not a NPE). Here is an example of a simple Image codec that takes responsibility for translating an Image into a correctly formed Bson Document:
The Custom Codec Example section of the documentation provides a better example. Ultimately, questions like these are better suited to our MongoDB community portal, located here. Or if you are a customer the MongoDB support portal, located at support.mongodb.com I feel that your questions are only showing part of the problem and trying to fix that. Should you follow up in one of the forums or via support, I recommend starting at first principles and describe what the data is in the JVM and how you would like it stored in MongoDB. The Codec system is flexible enough to meet users needs but it ulitmately boils down to converting data into valid Bson Documents. The downside to Custom Codecs are they can be cumbersome. I think / hope there should be a way to achieve what you desire without the need for a custom codecs - but I'm missing some part of the full problem. The default Kotlinx serialization based codec can handle conversion of these classes into Bson Documents, so part of the wider issue is missing. Please let me know if you followup via the community or the forums and link here, that way I can help there. Ross | ||||||||||||||||||||||||||||||||||||||
| Comment by Ross Lawley [ 09/Aug/23 ] | ||||||||||||||||||||||||||||||||||||||
As mentioned previously you can do the same with bson-kotlinx:
You will have to manually register the codec eg:
Ross | ||||||||||||||||||||||||||||||||||||||
| Comment by Uroš Jarc [ 08/Aug/23 ] | ||||||||||||||||||||||||||||||||||||||
|
Also this is probably related issue... I'm dealing with writting image to MongoDB and BsonBinary (Example in attachements) Test.kt
| ||||||||||||||||||||||||||||||||||||||
| Comment by Uroš Jarc [ 08/Aug/23 ] | ||||||||||||||||||||||||||||||||||||||
With that kind of approach, you lose all type-safety of compiled language... In the server I can register what kind of serializers I would want as the Json module with approprate serializers...
I don't see why would not MongoDB.kt library to the same? I think this issue is solved pretty neatly without creating breaking changes to the library API. | ||||||||||||||||||||||||||||||||||||||
| Comment by Ross Lawley [ 08/Aug/23 ] | ||||||||||||||||||||||||||||||||||||||
This should still possible with Kotlinx serialization - the KotlinSerializerCodec.create method allows you to provide a custom serializers module for the scope of serializing to Bson. However, kotlinx serialization always requires a key and value to be stored, so it may not be appropriate here. The following code shows that the HashedCodec can be used:
As Document is not a Serializable class the codec registry handles the encoding for all the values, as the serializable codec is lower down in the registry than the HashedCodec - it is choosen first. However, if the collection being used was not of MongoCollection<Document> but some annotated Serializable type eg: Collection<MySerializableType> then the Serialization framework would be used for all fields - unless there is an alternative Codec for MySerializableType higher up in the registry. Given this information - you should be able to either use a custom data class for storage in the database or write a Codec for the container class of Hashed as well. I hope that helps, Ross | ||||||||||||||||||||||||||||||||||||||
| Comment by Uroš Jarc [ 08/Aug/23 ] | ||||||||||||||||||||||||||||||||||||||
|
This should be possible to achive... @JvmInline @CustomCodex(with=EnctyptedCodec::class) | ||||||||||||||||||||||||||||||||||||||
| Comment by Uroš Jarc [ 08/Aug/23 ] | ||||||||||||||||||||||||||||||||||||||
|
ross@mongodb.com To use the same Serializer is not possible in my case because how I want to store in the database is different from the serialization that I want to use for the serialization of requests and responses... Those 2 serialization are not compatibly with each other... For the class to be contextual serialized I must marked my class with @Serializable and that breaks my codec that I wrote for the mongodb... Please reconsider my plea to still leave this issue in a bug mode...
| ||||||||||||||||||||||||||||||||||||||
| Comment by Ross Lawley [ 08/Aug/23 ] | ||||||||||||||||||||||||||||||||||||||
|
The bson-kotlinx library relies solely on the Kotlin Serialization framework for encoding / decoding. As such it once its being used it won't delegate to any codecs, rather it uses KSerializer instances for all types. In this instance I would recomment implementing your own contextual serializers for the Hashed class - see: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-core/kotlinx.serialization/ for more information. Note: Kotlinx serialization uses a compile time plugin to generate visitor code for serializable classes, where as the codec registry utilizes runtime look ups for finding codecs that handle serialization. As Kotlinx serialization doesn't have an escape route to break out of using its API for serializing certain types, integrating it with the codec registry for handling of sub types isn't seemingly possible. As such I think this is a case of "works as designed". Many thanks, Ross | ||||||||||||||||||||||||||||||||||||||
| Comment by Uroš Jarc [ 07/Aug/23 ] | ||||||||||||||||||||||||||||||||||||||
|
Full code is located in attachments! | ||||||||||||||||||||||||||||||||||||||
| Comment by PM Bot [ 07/Aug/23 ] | ||||||||||||||||||||||||||||||||||||||
|
Hi jar.fmf@gmail.com, thank you for reporting this issue! The team will look into it and get back to you soon. |