[JAVA-5292] Custom codec not found required by data class in list Created: 13/Jan/24  Updated: 24/Jan/24  Resolved: 24/Jan/24

Status: Closed
Project: Java Driver
Component/s: None
Affects Version/s: None
Fix Version/s: 5.0.0

Type: Bug Priority: Major - P3
Reporter: Carl Holwell Assignee: Ross Lawley
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Screenshot 2024-01-15 173428.png    
Documentation Changes: Not Needed
Documentation Changes Summary:

1. What would you like to communicate to the user about this feature?
2. Would you like the user to see examples of the syntax and/or executable code and its output?
3. Which versions of the driver/connector does this apply to?


 Description   

Summary

Given a data class Foo that has a property of type `kotlinx.datetime.LocalTime`, and another data class Bar that has a property of type `List<Foo>`

No codec is found for the `LocalTime` despite a custom codec being provided and working when no lists are involved.

 

Doesn't work

 

data class Foo(
    val time: LocalTime
)
 
data class Bar(
    val foos: List<Foo>
)
 

 

 

Works fine

 

*data class Foo(
    val time: LocalTime
)
 
data class Bar(
    val foo: Foo
)*

 

driver version 4.11.0

How to Reproduce

Use the above data classes and try to insert the `Bar` class to a collection with custom codec:

 

class LocalTimeCodec: Codec<LocalTime> {
 
    override fun encode(writer: BsonWriter, value: LocalTime, encoderContext: EncoderContext) {
        writer.writeString(value.toString())
    }
 
    override fun getEncoderClass(): Class<LocalTime> = LocalTime::class.java
 
    override fun decode(reader: BsonReader, decoderContext: DecoderContext): LocalTime =
        LocalTime.parse(reader.readString())
} 

 



 Comments   
Comment by Githook User [ 24/Jan/24 ]

Author:

{'name': 'Ross Lawley', 'email': 'ross@mongodb.com', 'username': 'rozza'}

Message: Handle kotlin / JVM erasure of types (#1295)

  • Handle kotlin / JVM erasure of types

It has been observed that the Kotlin can behave
differently during reflection and return Type<Object>
where normally it would report the correct type.

Checking for erasure allows the bson-kotlin library to handle
these cases and return the expected type.

JAVA-5292
Branch: master
https://github.com/mongodb/mongo-java-driver/commit/0d8cc808b489b850381efdbe195f3331df0837f6

Comment by Ross Lawley [ 16/Jan/24 ]

Filed: KT-65028 to track with Jet brains.

Comment by Ross Lawley [ 16/Jan/24 ]

Hi dev@noisysoftware.com,

Thanks for perserving, turns out its quite an interesting Kotlin bug. I have been able to reproduce with:

    data class Good(
        val duration: Int,
        val limits: List<Int>
    )
 
    data class Bad(
        val duration: Duration,
        val limits: List<Int>
    )

Good works and is happy and Bad leads to a CodecConfigurationException. Internally, the DataClassCodec uses reflection to determine container (List) types. This is where we can see Kotlin returning different results:

    Good::class.primaryConstructor?.parameters?.forEach { kParameter ->
        println(" :: ${kParameter.type.classifier} ${kParameter.type.arguments} : ${kParameter.type.arguments.mapNotNull { it.type?.javaType }.toList()}")
    }
 
    println("--------------")
 
    Bad::class.primaryConstructor?.parameters?.forEach { kParameter ->
        println(" :: ${kParameter.type.classifier} ${kParameter.type.arguments} : ${kParameter.type.arguments.mapNotNull { it.type?.javaType }.toList()}")
    }

Outputs:

 :: class kotlin.Int [] : []
 :: class kotlin.collections.List [kotlin.Int] : [class java.lang.Integer] // No Erasure!
--------------
 :: class kotlin.time.Duration [] : []
 :: class kotlin.collections.List [kotlin.Int] : [class java.lang.Object] // Erased?!

Notice the javaType of the type arguments are erased for some reason in Bad but not in Good.

Ross

Comment by Carl Holwell [ 15/Jan/24 ]

Not working example

import com.mongodb.MongoClientSettings.getDefaultCodecRegistry
import com.mongodb.kotlin.client.coroutine.MongoClient
import com.noisy.leq.model.LeqTType
import com.noisy.leq.model.Weighting
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.runBlocking
import org.bson.BsonReader
import org.bson.BsonWriter
import org.bson.codecs.Codec
import org.bson.codecs.DecoderContext
import org.bson.codecs.EncoderContext
import org.bson.codecs.configuration.CodecRegistries.fromCodecs
import org.bson.codecs.configuration.CodecRegistries.fromRegistries
import org.bson.types.ObjectId
import kotlin.time.Duration
import kotlin.time.Duration.Companion.minutes
 
/**
 * [Duration] codec
 */
class DurationCodec : Codec<Duration> {
 
    override fun encode(writer: BsonWriter, value: Duration, encoderContext: EncoderContext) {
        writer.writeString(value.toIsoString())
    }
 
    override fun getEncoderClass(): Class<Duration> = Duration::class.java
 
    override fun decode(reader: BsonReader, decoderContext: DecoderContext): Duration =
        Duration.parseIsoString(reader.readString())
}
 
data class Limit(
    val value: Double?,
    val tolerance: Double?,
    val from: Int,
    val to: Int
)
 
data class Arguments(
    val _id: ObjectId = ObjectId(),
    val _venueId: ObjectId,
    val _soundLevelMeterId: ObjectId,
    val duration: Duration,
    val weighting: Weighting,
    val type: LeqTType,
    val limits: List<Limit>
)
 
runBlocking {
 
    val mongoClient = MongoClient.create("mongodb://localhost:27017")
    val db = mongoClient.getDatabase("test")
        .withCodecRegistry(fromRegistries(getDefaultCodecRegistry(), fromCodecs(DurationCodec())))
 
    val collection = db.getCollection<Arguments>("test")
    
    val leqt = Arguments(
        _venueId = ObjectId(),
        _soundLevelMeterId = ObjectId(),
        weighting = Weighting.A,
        type = LeqTType.MOVING,
        duration = 1.minutes,
        limits = listOf(
            Limit(
                value = 80.0,
                tolerance = 3.0,
                from = 0,
                to = 4
            ),
            Limit(
                value = 85.0,
                tolerance = 3.0,
                from = 4,
                to = 11
            )
        )
    )
 
    println(collection.insertOne(leqt))
    println(collection.find().first())
 
    db.drop()
    mongoClient.close()
} 

Comment by Carl Holwell [ 15/Jan/24 ]

Okay looks like LocalTime was a red herring, and I still receive the same error if I represent the time as an Int

Actual data classes: 

data class Arguments( //what I'm trying to insert
    val _id: ObjectId = ObjectId(),
    val _venueId: ObjectId,
    val _soundLevelMeterId: ObjectId,
    val weighting: Weighting, //enum
    val duration: Duration, //kotlin.time.Duration
    val type: LeqTType, //enum
    val limits: List<Limit>
) 
 
data class Limit(
    val value: Double?,
    val tolerance: Double?,
    val from: Int,
    val to: Int
)

Comment by Carl Holwell [ 15/Jan/24 ]

Full stacktrace 

org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for CodecCacheKey{clazz=class java.lang.Object, types=null}.
    at org.bson.internal.ProvidersCodecRegistry.lambda$get$0(ProvidersCodecRegistry.java:87)
    at java.base/java.util.Optional.orElseGet(Optional.java:364)
    at org.bson.internal.ProvidersCodecRegistry.get(ProvidersCodecRegistry.java:80)
    at org.bson.internal.ChildCodecRegistry.get(ChildCodecRegistry.java:68)
    at org.bson.codecs.ContainerCodecHelper.getCodec(ContainerCodecHelper.java:71)
    at org.bson.codecs.CollectionCodecProvider.get(CollectionCodecProvider.java:99)
    at org.bson.internal.ProvidersCodecRegistry.getFromCodecProvider(ProvidersCodecRegistry.java:95)
    at org.bson.internal.ProvidersCodecRegistry.get(ProvidersCodecRegistry.java:70)
    at org.bson.internal.ProvidersCodecRegistry.getFromCodecProvider(ProvidersCodecRegistry.java:95)
    at org.bson.internal.ProvidersCodecRegistry.get(ProvidersCodecRegistry.java:70)
    at org.bson.internal.ProvidersCodecRegistry.getFromCodecProvider(ProvidersCodecRegistry.java:95)
    at org.bson.codecs.configuration.OverridableUuidRepresentationCodecProvider.get(OverridableUuidRepresentationCodecProvider.java:47)
    at org.bson.internal.ProvidersCodecRegistry.getFromCodecProvider(ProvidersCodecRegistry.java:95)
    at org.bson.internal.ProvidersCodecRegistry.lambda$get$0(ProvidersCodecRegistry.java:82)
    at java.base/java.util.Optional.orElseGet(Optional.java:364)
    at org.bson.internal.ProvidersCodecRegistry.get(ProvidersCodecRegistry.java:80)
    at org.bson.internal.ChildCodecRegistry.get(ChildCodecRegistry.java:81)
    at org.bson.codecs.kotlin.DataClassCodec$Companion.getCodec(DataClassCodec.kt:226)
    at org.bson.codecs.kotlin.DataClassCodec$Companion.getCodec(DataClassCodec.kt:197)
    at org.bson.codecs.kotlin.DataClassCodec$Companion.create$bson_kotlin(DataClassCodec.kt:148)
    at org.bson.codecs.kotlin.DataClassCodecProvider.get(DataClassCodecProvider.kt:28)
    at org.bson.internal.ProvidersCodecRegistry.getFromCodecProvider(ProvidersCodecRegistry.java:95)
    at com.mongodb.KotlinCodecProvider.get(KotlinCodecProvider.java:83)
    at org.bson.internal.ProvidersCodecRegistry.getFromCodecProvider(ProvidersCodecRegistry.java:95)
    at org.bson.internal.ProvidersCodecRegistry.get(ProvidersCodecRegistry.java:70)
    at org.bson.internal.ProvidersCodecRegistry.getFromCodecProvider(ProvidersCodecRegistry.java:95)
    at org.bson.internal.ProvidersCodecRegistry.get(ProvidersCodecRegistry.java:70)
    at org.bson.internal.ProvidersCodecRegistry.getFromCodecProvider(ProvidersCodecRegistry.java:95)
    at org.bson.codecs.configuration.OverridableUuidRepresentationCodecProvider.get(OverridableUuidRepresentationCodecProvider.java:47)
    at org.bson.internal.ProvidersCodecRegistry.getFromCodecProvider(ProvidersCodecRegistry.java:95)
    at org.bson.internal.ProvidersCodecRegistry.lambda$get$0(ProvidersCodecRegistry.java:82)
    at java.base/java.util.Optional.orElseGet(Optional.java:364)
    at org.bson.internal.ProvidersCodecRegistry.get(ProvidersCodecRegistry.java:80)
    at org.bson.internal.ProvidersCodecRegistry.get(ProvidersCodecRegistry.java:50)
    at com.mongodb.internal.operation.Operations.createFindOperation(Operations.java:186)
    at com.mongodb.internal.operation.Operations.find(Operations.java:176)
    at com.mongodb.internal.operation.AsyncOperations.find(AsyncOperations.java:122)
    at com.mongodb.reactivestreams.client.internal.FindPublisherImpl.asAsyncReadOperation(FindPublisherImpl.java:222)
    at com.mongodb.reactivestreams.client.internal.FindPublisherImpl.asAsyncReadOperation(FindPublisherImpl.java:38)
    at com.mongodb.reactivestreams.client.internal.BatchCursorPublisher.lambda$batchCursor$3(BatchCursorPublisher.java:129)
    at com.mongodb.reactivestreams.client.internal.MongoOperationPublisher.createReadOperationMono(MongoOperationPublisher.java:418)
    at com.mongodb.reactivestreams.client.internal.MongoOperationPublisher.createReadOperationMono(MongoOperationPublisher.java:411)
    at com.mongodb.reactivestreams.client.internal.BatchCursorPublisher.batchCursor(BatchCursorPublisher.java:133)
    at com.mongodb.reactivestreams.client.internal.BatchCursorPublisher.batchCursor(BatchCursorPublisher.java:129)
    at com.mongodb.reactivestreams.client.internal.BatchCursorFlux.lambda$subscribe$1(BatchCursorFlux.java:53)
    at reactor.core.publisher.FluxCreate$BaseSink.onRequest(FluxCreate.java:557)
    at reactor.core.publisher.FluxCreate$SerializedFluxSink.onRequest(FluxCreate.java:271)
    at com.mongodb.reactivestreams.client.internal.BatchCursorFlux.lambda$subscribe$2(BatchCursorFlux.java:47)
    at reactor.core.publisher.FluxCreate.subscribe(FluxCreate.java:95)
    at reactor.core.publisher.Flux.subscribe(Flux.java:8660)
    at com.mongodb.reactivestreams.client.internal.BatchCursorFlux.subscribe(BatchCursorFlux.java:73)
    at com.mongodb.reactivestreams.client.internal.BatchCursorPublisher.subscribe(BatchCursorPublisher.java:125)
    at kotlinx.coroutines.reactive.PublisherAsFlow.collectImpl(ReactiveFlow.kt:94)
    at kotlinx.coroutines.reactive.PublisherAsFlow.collect(ReactiveFlow.kt:79)
    at com.mongodb.kotlin.client.coroutine.FindFlow.collect(FindFlow.kt:296)
    at kotlinx.coroutines.flow.FlowKt__CollectionKt.toCollection(Collection.kt:26)
    at kotlinx.coroutines.flow.FlowKt.toCollection(Unknown Source)
    at kotlinx.coroutines.flow.FlowKt__CollectionKt.toList(Collection.kt:15)
    at kotlinx.coroutines.flow.FlowKt.toList(Unknown Source)
    at kotlinx.coroutines.flow.FlowKt__CollectionKt.toList$default(Collection.kt:15)
    at kotlinx.coroutines.flow.FlowKt.toList$default(Unknown Source)
    at com.noisy.leq.data.leqt_database.LeqTDatabase$findAll$2.invokeSuspend(LeqTDatabase.kt:63)
    at com.noisy.leq.data.leqt_database.LeqTDatabase$findAll$2.invoke(LeqTDatabase.kt)
    at com.noisy.leq.data.leqt_database.LeqTDatabase$findAll$2.invoke(LeqTDatabase.kt)
    at com.noisy.service_foundation.mongo.MongoKt$run$2.invokeSuspend(Mongo.kt:86)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
    at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
    at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684) 

Comment by Carl Holwell [ 15/Jan/24 ]

Hey ross@mongodb.com , thanks for the quick response

I am registering multiple custom codecs at the same time:

 

val codecs = fromCodecs(InstantCodec(), DurationCodec(), LocalTimeCodec())
val codecRegistry = fromRegistries(codecs, getDefaultCodecRegistry())
 
val mongoDatabase = mongoClient
    .getDatabase(mongodbConfig.database.name)
    .withCodecRegistry(codecRegistry) 

 

and when debugging where this exception is thrown, in the cache I can see the InstantCodec and DurationCodec are but not LocalTimeCodec...

The exception is actually different in that it is trying to find a codec for Object rather than LocalTime, however what led me to this conclusion was changing the data type from a list to just LocalTime works.

 Can't find a codec for CodecCacheKey{clazz=class java.lang.Object, types=null}

 

Thanks

Comment by Ross Lawley [ 15/Jan/24 ]

Hi dev@noisysoftware.com,

Thanks for the ticket. I've tested with the following code and it works as expected:

package example
 
import com.mongodb.MongoClientSettings
import com.mongodb.kotlin.client.MongoClient
import org.bson.BsonReader
import org.bson.BsonWriter
import org.bson.Document
import org.bson.codecs.Codec
import org.bson.codecs.DecoderContext
import org.bson.codecs.EncoderContext
import org.bson.codecs.configuration.CodecRegistries.fromCodecs
import org.bson.codecs.configuration.CodecRegistries.fromRegistries
import kotlinx.datetime.LocalTime
 
 
data class Foo(
    val time: LocalTime
)
 
data class Bar(
    val foos: List<Foo>
)
 
 
class LocalTimeCodec: Codec<LocalTime> {
 
    override fun encode(writer: BsonWriter, value: LocalTime, encoderContext: EncoderContext) {
        writer.writeString(value.toString())
    }
 
    override fun getEncoderClass(): Class<LocalTime> = LocalTime::class.java
 
    override fun decode(reader: BsonReader, decoderContext: DecoderContext): LocalTime =
        LocalTime.parse(reader.readString())
}
 
 
fun main() {
 
    val mongoClient = MongoClient.create("mongodb://localhost/")
    val db = mongoClient.getDatabase("test")
        .withCodecRegistry(fromRegistries(MongoClientSettings.getDefaultCodecRegistry(), fromCodecs(LocalTimeCodec()))) // Register the LocalTimeCodec
 
 
    val collection = db.getCollection<Bar>("test")
    collection.drop()
 
    val bar = Bar(listOf(Foo(time = LocalTime.fromSecondOfDay(0)), Foo(time = LocalTime.fromSecondOfDay(43_200))))
 
    println(collection.insertOne(bar))
    println(collection.find().first())
    println(collection.withDocumentClass<Document>().find().first().toJson())
 
    db.drop()
    mongoClient.close()
}

Outputs:

AcknowledgedInsertOneResult{insertedId=BsonObjectId{value=65a508100c7fbc41681618f6}}
Bar(foos=[Foo(time=00:00), Foo(time=12:00)])
{"_id": {"$oid": "65a508100c7fbc41681618f6"}, "foos": [{"time": "00:00"}, {"time": "12:00"}]}

Only if I fail to register the Codec do I see a Can't find a codec for CodecCacheKey{clazz=class kotlinx.datetime.LocalTime, types=null}. exception. For more information see the Codecs section of the documentation.

I hope that solves the issue for you.

Ross

Comment by PM Bot [ 13/Jan/24 ]

Hi dev@noisysoftware.com, thank you for reporting this issue! The team will look into it and get back to you soon.

Generated at Thu Feb 08 09:04:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.