<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 09:05:56 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[KAFKA-246] Support namespaces provided in the Schema</title>
                <link>https://jira.mongodb.org/browse/KAFKA-246</link>
                <project id="16285" key="KAFKA">Kafka Connector</project>
                    <description>&lt;p&gt;See: &lt;a href=&quot;https://www.mongodb.com/community/forums/t/kafka-source-connector-output-schema-value-not-registering-namespace/102645/2&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://www.mongodb.com/community/forums/t/kafka-source-connector-output-schema-value-not-registering-namespace/102645/2&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;We took the default schema from the source connector documentation under &lt;tt&gt;output.schema.value&lt;/tt&gt; and modified it as following:&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;Adding namespace&lt;/li&gt;
	&lt;li&gt;Renaming ChangeStream to MongoSourceChangeEvent&lt;/li&gt;
&lt;/ul&gt;



&lt;p&gt;We use the schema above in output.schema.value. When the schema is registered by the connector in the schema registry - the namespace field is not included.&lt;/p&gt;

&lt;p&gt;This creates a problem for us in Kotlin, where we can&#8217;t include namespace in the import because deserialization will fail, instead of:&lt;br/&gt;
&lt;tt&gt;import com.company.common.avro.MongoSourceChangeEvent&lt;/tt&gt;&lt;/p&gt;

&lt;p&gt;We are forced to remove the namespace from the schema and import it without namespace (the schema being registered is also being used to auto generate Avro classes) :&lt;br/&gt;
&lt;tt&gt;import MongoSourceChangeEvent&lt;/tt&gt;&lt;/p&gt;

&lt;p&gt;We want to have a proper namespace for our auto generated classes and currently this issue prevents us from doing so.&lt;/p&gt;&lt;/blockquote&gt;</description>
                <environment></environment>
        <key id="1848874">KAFKA-246</key>
            <summary>Support namespaces provided in the Schema</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="10300" iconUrl="https://jira.mongodb.org/images/icons/priorities/medium.svg">Unknown</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="valentin.kovalenko@mongodb.com">Valentin Kavalenka</assignee>
                                    <reporter username="ross@mongodb.com">Ross Lawley</reporter>
                        <labels>
                            <label>size-small</label>
                    </labels>
                <created>Wed, 11 Aug 2021 14:30:27 +0000</created>
                <updated>Sat, 28 Oct 2023 10:46:13 +0000</updated>
                            <resolved>Wed, 16 Feb 2022 15:08:40 +0000</resolved>
                                                    <fixVersion>1.7.0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="4358721" author="xgen-internal-githook" created="Wed, 16 Feb 2022 15:08:05 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Valentin Kovalenko&apos;, &apos;email&apos;: &apos;valentin.kovalenko@mongodb.com&apos;, &apos;username&apos;: &apos;stIncMale&apos;}
&lt;p&gt;Message: Correctly process namespaces when converting `org.apache.avro.Schema` to `org.apache.kafka.connect.data.Schema` (#103)&lt;/p&gt;

&lt;p&gt;`org.apache.kafka.connect.data.Schema` does not separate a namespace from a short name,&lt;br/&gt;
and always uses full names, while `org.apache.avro.Schema` separates them.&lt;br/&gt;
The &quot;always uses full names&quot; part about `org.apache.kafka.connect.data.Schema`&lt;br/&gt;
is not documented anywhere, but based on the following example&lt;br/&gt;
&lt;a href=&quot;https://docs.confluent.io/platform/current/tutorials/examples/connect-streams-pipeline/docs/index.html#example-3-jdbc-source-connector-with-specificavro-key-string-null-and-value-specificavro&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://docs.confluent.io/platform/current/tutorials/examples/connect-streams-pipeline/docs/index.html#example-3-jdbc-source-connector-with-specificavro-key-string-null-and-value-specificavro&lt;/a&gt;&lt;br/&gt;
and the `SetSchemaMetadata`&lt;br/&gt;
(&lt;a href=&quot;https://github.com/apache/kafka/blob/trunk/connect/transforms/src/main/java/org/apache/kafka/connect/transforms/SetSchemaMetadata.java&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/apache/kafka/blob/trunk/connect/transforms/src/main/java/org/apache/kafka/connect/transforms/SetSchemaMetadata.java&lt;/a&gt;),&lt;br/&gt;
it is clear that the name in `org.apache.kafka.connect.data.Schema`&lt;br/&gt;
is supposed to be the full name.&lt;/p&gt;

&lt;p&gt;Avro supports namespaces only for schemas of records, enums, and fixed types,&lt;br/&gt;
of which MongoDB Kafka Connector supports only records.&lt;br/&gt;
Therefore, we need to care about using `org.apache.avro.Schema.getFullName`&lt;br/&gt;
as `org.apache.kafka.connect.data.Schema.name()` only for record schemas.&lt;/p&gt;

&lt;p&gt;It is worth pointing out that as a result of this change, MongoDB Kafka Connector&lt;br/&gt;
will start storing a bit different Avro schemas in Confluent Schema Registry&lt;br/&gt;
via `AvroConverter`&lt;br/&gt;
(&lt;a href=&quot;https://github.com/confluentinc/schema-registry/blob/master/avro-converter/src/main/java/io/confluent/connect/avro/AvroConverter.java&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/confluentinc/schema-registry/blob/master/avro-converter/src/main/java/io/confluent/connect/avro/AvroConverter.java&lt;/a&gt;).&lt;br/&gt;
This is not supposed to cause any compatibility issues for connectors, judging from&lt;br/&gt;
&lt;a href=&quot;https://docs.confluent.io/platform/current/schema-registry/avro.html#schema-evolution-and-compatibility&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://docs.confluent.io/platform/current/schema-registry/avro.html#schema-evolution-and-compatibility&lt;/a&gt;,&lt;br/&gt;
but will be visible for those users who explicitly fetch schemas from Schema Registry.&lt;br/&gt;
It is unclear why users would do that and how, but apparently it is what the user&lt;br/&gt;
reported &lt;a href=&quot;https://jira.mongodb.org/browse/KAFKA-246&quot; title=&quot;Support namespaces provided in the Schema&quot; class=&quot;issue-link&quot; data-issue-key=&quot;KAFKA-246&quot;&gt;&lt;del&gt;KAFKA-246&lt;/del&gt;&lt;/a&gt; is doing. Here are some thoughts on how a user may fetch relevant schemas from Schema Registry:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;`AvroConverter.fromConnectData` encodes the ID of a registered schema as part of the data&lt;br/&gt;
it produces, which is then stored as key/value of a Kafka record. A user may decode this ID from the raw data&lt;br/&gt;
and use it to fetch the schema by ID (`SchemaRegistryClient.getSchemaById`).&lt;/li&gt;
	&lt;li&gt;`SchemaRegistryClient.getLatestSchemaMetadata(&quot;subject&quot;).getSchema()`, where subject is&lt;br/&gt;
determined by the subject name strategy&lt;br/&gt;
(&lt;a href=&quot;https://docs.confluent.io/platform/current/schema-registry/serdes-develop/index.html#sr-schemas-subject-name-strategy&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://docs.confluent.io/platform/current/schema-registry/serdes-develop/index.html#sr-schemas-subject-name-strategy&lt;/a&gt;).&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Other useful links:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;&lt;a href=&quot;https://docs.confluent.io/platform/current/schema-registry/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://docs.confluent.io/platform/current/schema-registry/&lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://avro.apache.org/docs/current/spec.html#names&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://avro.apache.org/docs/current/spec.html#names&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/KAFKA-246&quot; title=&quot;Support namespaces provided in the Schema&quot; class=&quot;issue-link&quot; data-issue-key=&quot;KAFKA-246&quot;&gt;&lt;del&gt;KAFKA-246&lt;/del&gt;&lt;/a&gt;&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo-kafka/commit/6872a7e05fbde4dbc04098f90598c168f0694948&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo-kafka/commit/6872a7e05fbde4dbc04098f90598c168f0694948&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4213823" author="JIRAUSER1263261" created="Mon, 29 Nov 2021 04:16:08 +0000"  >&lt;p&gt;Hi @Ross Lawley/@Esha Bhargava, could I submit a proposed fix to this issue via a pull request?&lt;/p&gt;

&lt;p&gt;I think we can include the namespace from &lt;tt&gt;org.apache.avro.Schema&lt;/tt&gt; as a part of &lt;tt&gt;org.apache.kafka.connect.data.Schema&lt;/tt&gt;&apos;s name field to ensure the namespace is not omitted from the &lt;tt&gt;org.apache.kafka.connect.data.Schema&lt;/tt&gt; produced by &lt;tt&gt;com.mongodb.kafka.connect.source.schema.AvroSchema&lt;/tt&gt;&apos;s &lt;tt&gt;createSchema&lt;/tt&gt; method.&#160;&lt;/p&gt;

&lt;p&gt;This practice is also used in Confluent&apos;s &lt;tt&gt;io.confluent.connect.avro.AvroData&lt;/tt&gt;:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://github.com/confluentinc/schema-registry/blob/master/avro-data/src/main/java/io/confluent/connect/avro/AvroData.java#L786&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/confluentinc/schema-registry/blob/master/avro-data/src/main/java/io/confluent/connect/avro/AvroData.java#L786&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Thanks.&lt;/p&gt;</comment>
                            <comment id="4009012" author="ross@10gen.com" created="Thu, 19 Aug 2021 08:13:46 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=guptabrijmohan30%40gmail.com&quot; class=&quot;user-hover&quot; rel=&quot;guptabrijmohan30@gmail.com&quot;&gt;guptabrijmohan30@gmail.com&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;I don&apos;t think a PR has been created for this feature yet.  Its not planned for release this quarter.&lt;/p&gt;

&lt;p&gt;Regarding the test failure, you&apos;d have to look at the test report to understand more about what failed and why.&lt;/p&gt;

&lt;p&gt;Ross&lt;/p&gt;</comment>
                            <comment id="4008531" author="JIRAUSER1261830" created="Thu, 19 Aug 2021 04:29:23 +0000"  >&lt;p&gt;Hi @Ross Lawley,&lt;/p&gt;

&lt;p&gt;Please let me know when you are targeting this release? Because I am not able to build given PR.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10320">
                    <name>Documented</name>
                                                                <inwardlinks description="is documented by">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                        <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[5002K00000xDkf0QAC]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10257" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Documentation Changes</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10250"><![CDATA[Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_21553" key="com.atlassian.jira.plugin.system.customfieldtypes:labels">
                        <customfieldname>Quarter</customfieldname>
                        <customfieldvalues>
                                        <label>FY22Q4</label>
    
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr3ngv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                </customfields>
    </item>
</channel>
</rss>