<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 09:00:20 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[JAVA-3747] Redesign Codec Resolution</title>
                <link>https://jira.mongodb.org/browse/JAVA-3747</link>
                <project id="10006" key="JAVA">Java Driver</project>
                    <description>&lt;p&gt;The MongoDB Scala driver advertises that it is a &quot;modern idiomatic MongoDB Scala Driver&quot;. I would mostly agree, if there wasn&apos;t one big issue: The codec resolution is done at runtime via reflection. This is not idiomatic scala and has limitations. We can do that better &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/biggrin.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/thumbs_up.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/p&gt;

&lt;p&gt;One of the biggest drawbacks of the current approach is that the jvm does not provide type information for Generic types at runtime. We can therefore not properly deserialize a List&lt;span class=&quot;error&quot;&gt;&amp;#91;String&amp;#93;&lt;/span&gt; since the information that the List contains String is not available via the reflection api.&lt;/p&gt;

&lt;p&gt;Another Issue is that a codec for class that may some how be contained in the class-structure of the elements store in the Database must be explicitly registered. If I for instance have a `case class Foo(a: Either&lt;span class=&quot;error&quot;&gt;&amp;#91;Bar, Baz&amp;#93;&lt;/span&gt;)`, I must register a Codec for the case classes Foo, Bar, Baz and the types of any fields of their fields.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;TheSolution&quot;&gt;&lt;/a&gt;The Solution&lt;/h4&gt;

&lt;p&gt;A more idiomatic scala way to implement codec resolution would be to use implicits instead. This would for instance solve the issue of deserializing Lists, as generic types are available at compile time. There will even no longer be any runtime overhead for the lookup up of codec classes in the CodecRegistries.&lt;/p&gt;

&lt;p&gt;To implement that, the Driver should provide a Generic Codec&lt;span class=&quot;error&quot;&gt;&amp;#91;A&amp;#93;&lt;/span&gt; class. When creating for instance an MongoCollection&lt;span class=&quot;error&quot;&gt;&amp;#91;A&amp;#93;&lt;/span&gt;, a Codec&lt;span class=&quot;error&quot;&gt;&amp;#91;A&amp;#93;&lt;/span&gt; class would be implicitly resolved. There is no longer a need for manually registering codecs.&lt;/p&gt;

&lt;p&gt;For me this is one of the most anoying aspects when working with Mongo on the JVM. While I&apos;ve currently decided to just no longer use Mongo when writing Software for the JVM, I&apos;ve written a small/incomplete wrapper around the Mongo java driver in the past. This incomplete wrapper implements Codec resolution at compile time using implicits and can serialize Lists and Map&lt;span class=&quot;error&quot;&gt;&amp;#91;String, ?&amp;#93;&lt;/span&gt; properly. It can even serializes any &quot;case class&quot; or ADT (sealed trait + case class implementations) data structures using shapeless, without defining or registering codecs somewhere. A cust om written Macro as currently used could make this even more performant.&lt;/p&gt;

&lt;p&gt;This wrapper (see attachment) is far from beeing an ideal implementation and was never used seriously, but take it as an inspiration of what the future Mongo-Scala-Driver might look like. You may also take at the Couchbase Scala Driver. Their Scala driver and its serialization mechanism feels way more idiomatic!&lt;/p&gt;

&lt;p&gt;Could this be a potential candidate for next major release of the Mongo-Scala-Driver?&lt;/p&gt;</description>
                <environment></environment>
        <key id="1363619">JAVA-3747</key>
            <summary>Redesign Codec Resolution</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="aki-ks.cloud.mongodb.com@m.mader.dev">Aki ks</reporter>
                        <labels>
                    </labels>
                <created>Wed, 27 May 2020 13:49:56 +0000</created>
                <updated>Wed, 30 Mar 2022 23:13:57 +0000</updated>
                                                                            <component>Codecs</component>
                    <component>Scala</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="3203154" author="ross@10gen.com" created="Wed, 10 Jun 2020 14:36:32 +0000"  >&lt;p&gt;Thanks again &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=aki-ks.cloud.mongodb.com%40m.mader.dev&quot; class=&quot;user-hover&quot; rel=&quot;aki-ks.cloud.mongodb.com@m.mader.dev&quot;&gt;aki-ks.cloud.mongodb.com@m.mader.dev&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Some great points. I&apos;m moving this ticket to &quot;open&quot; for future consideration.&lt;/p&gt;</comment>
                            <comment id="3175570" author="aki-ks.cloud.mongodb.com@m.mader.dev" created="Tue, 2 Jun 2020 11:09:30 +0000"  >&lt;p&gt;The current scala driver already includes Macros that work for case classes and adt&apos;s, but those do only summon one `Codec&lt;span class=&quot;error&quot;&gt;&amp;#91;A&amp;#93;&lt;/span&gt;` for the one requested type &quot;A&quot;. Instead of using shapeless we should implement such custom macros. It would generate more optimized Code the when using Shapleless. Back then I&apos;ve just decided to use Shapeless since I&apos;ve wanted to get running Code quickly and I&apos;ve often had troubled with scala macros in the past.&lt;/p&gt;

&lt;p&gt;So once you&apos;ve retrieved macro-generated codecs, you must currently register them in the Codec-Registry one by one. For every type that might be contained in the Datastructure to be serialized, one Codec must get explicitly registered. As you&apos;ve said, at runtime some kind of `Map[Class&lt;span class=&quot;error&quot;&gt;&amp;#91;A&amp;#93;&lt;/span&gt;, Codec&lt;span class=&quot;error&quot;&gt;&amp;#91;A&amp;#93;&lt;/span&gt;]` looks those Codes up. But if you&apos;ve for instance forgot to register a Codec for one type, you will notice this only at runtime when such a type should get serialized for the first time.&lt;br/&gt;
...And If I remember correctly from what I&apos;ve seen a few years ago: If CodecRegistries and CodecProviders are combined in a bad way, which can happen quite easily, a list of codecs gets iterated and one after another gets asked whether it supports the requested class (linear-time) rather than using one (constant-time) HashMap lookup.&lt;/p&gt;

&lt;p&gt;&lt;ins&gt;My main idea is to completely get rid of the usage of CodecRegistry in the Scala driver&lt;/ins&gt;. Due to limitiations in the Java language it was required there and it has &lt;ins&gt;many&lt;/ins&gt; disadvantages, but with the power of implicits we do no longer need to rely on it for codec-resolution in the Scala-driver.&lt;br/&gt;
 When ever a database operation such as `def find&lt;span class=&quot;error&quot;&gt;&amp;#91;A&amp;#93;&lt;/span&gt;` or `def updateOne&lt;span class=&quot;error&quot;&gt;&amp;#91;A&amp;#93;&lt;/span&gt;` is invoked, a Typeparameter is available that allows to implicitly find the corresponding Codec at Compiletime.&lt;/p&gt;</comment>
                            <comment id="3175418" author="ross@10gen.com" created="Tue, 2 Jun 2020 08:38:32 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=aki-ks.cloud.mongodb.com%40m.mader.dev&quot; class=&quot;user-hover&quot; rel=&quot;aki-ks.cloud.mongodb.com@m.mader.dev&quot;&gt;aki-ks.cloud.mongodb.com@m.mader.dev&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thanks for the ticket.&#160; As you may know the Scala driver is a layer written upon the Java driver and internally, the Java driver uses a CodecRegistry for converting type T to its equivalent Bson Type.&#160; The registry can be thought as a Map like structure where the key is the class of the type to be encoded and the value is the Codec.&#160; So it isn&apos;t really reflection based persae but as you rightly pointed out that can cause issues where generics are involved - as the type information is erased the user has to cast to the expected type.&lt;/p&gt;


&lt;p&gt;To combat this the Scala driver does two things:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;Implicitly wraps the strongly typed BsonDocument into a Scala Document class.&lt;br/&gt;
Adding support for implicit conversion from Scala native types to Bson types&lt;br/&gt;
Has the base type of BsonValues so no cast class exceptions&#160;&lt;/li&gt;
	&lt;li&gt;The Macro Codec&lt;br/&gt;
Adds support for Case Classes by looking up type information at compile time.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;That said, I really like what you have done by using shapeless you&apos;ve used reflection to implicitly handle case classes and other scala types to map into a Codec.&#160; Also by wrapping that into a Macro to set the Codec at compile time that would improve the performance by removing the runtime reflection.&lt;/p&gt;

&lt;p&gt;These are two approaches to the same problem, the Java driver uses a single registry throughout and your approach provides a specialized custom codec for each usecase.&lt;/p&gt;

&lt;p&gt;The only downside would be to take a hard dependency on shapeless, which can causes issues with users ecosystems with different versions etc.&#160; The JVM drivers aim to take the absolute minimum of outside dependencies because of the drivers wide usecases and the interactions with many different systems and stacks.&lt;/p&gt;

&lt;p&gt;I would love to see an alternative Scala driver implementation written ontop of the Reactive Streams Driver that uses shapeless and your ideas.&#160; I think that could be a benefit to Scala users.&lt;/p&gt;

&lt;p&gt;Ross&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="262665" name="mongo.zip" size="6751" author="aki-ks.cloud.mongodb.com@m.mader.dev" created="Wed, 27 May 2020 13:41:53 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr6tcv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            </customfields>
    </item>
</channel>
</rss>