<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 09:05:27 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[KAFKA-48] Can&apos;t set custom message key</title>
                <link>https://jira.mongodb.org/browse/KAFKA-48</link>
                <project id="16285" key="KAFKA">Kafka Connector</project>
                    <description>&lt;p&gt;I try to use mongo document id as message key. Following this examples&#160;&lt;a href=&quot;https://docs.confluent.io/current/connect/transforms/valuetokey.html#valuetokey&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://docs.confluent.io/current/connect/transforms/valuetokey.html#valuetokey&lt;/a&gt;&#160;i specify settings in MongoSourceConnector.properties but connector raise exception:&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Only Struct objects supported for &lt;span class=&quot;error&quot;&gt;&amp;#91;copying fields from value to key&amp;#93;&lt;/span&gt;, found: java.lang.String&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;&#160;&lt;/p&gt;</description>
                <environment></environment>
        <key id="871487">KAFKA-48</key>
            <summary>Can&apos;t set custom message key</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13202">Works as Designed</resolution>
                                        <assignee username="ross@mongodb.com">Ross Lawley</assignee>
                                    <reporter username="industrial.lang@gmail.com">Dionis Stepanidis</reporter>
                        <labels>
                    </labels>
                <created>Tue, 23 Jul 2019 14:05:30 +0000</created>
                <updated>Fri, 27 Oct 2023 11:54:16 +0000</updated>
                            <resolved>Thu, 25 Jul 2019 07:38:06 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="2343011" author="ross@10gen.com" created="Thu, 25 Jul 2019 07:38:06 +0000"  >&lt;p&gt;I&apos;m closing as Works as Designed - only because Kafka and the connector are working as they currently are supposed to.  &lt;/p&gt;

&lt;p&gt;Please follow / watch &lt;a href=&quot;https://jira.mongodb.org/browse/KAFKA-40&quot; title=&quot;Support schema for Source connector&quot; class=&quot;issue-link&quot; data-issue-key=&quot;KAFKA-40&quot;&gt;&lt;del&gt;KAFKA-40&lt;/del&gt;&lt;/a&gt; for updates about supporting schema&apos;d data from the source connector, which will then allow you to use Kafka&apos;s value to key SMT.&lt;/p&gt;

&lt;p&gt;Ross&lt;/p&gt;</comment>
                            <comment id="2341416" author="industrial.lang@gmail.com" created="Wed, 24 Jul 2019 14:00:13 +0000"  >&lt;blockquote&gt;
&lt;p&gt;If you are using the Mongo Kafka Connector as the source and want to apply this SMT then you will have to wait until &lt;a href=&quot;https://jira.mongodb.org/browse/KAFKA-40&quot; title=&quot;Support schema for Source connector&quot; class=&quot;issue-link&quot; data-issue-key=&quot;KAFKA-40&quot;&gt;&lt;del&gt;KAFKA-40&lt;/del&gt;&lt;/a&gt; to support schema&apos;d data from the source.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;I will wait that feature. Thanks for help Ross&lt;/p&gt;</comment>
                            <comment id="2341380" author="ross@10gen.com" created="Wed, 24 Jul 2019 13:46:42 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=industrial.lang%40gmail.com&quot; class=&quot;user-hover&quot; rel=&quot;industrial.lang@gmail.com&quot;&gt;industrial.lang@gmail.com&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;It depends how you are using the SMT feature.  If you are applying it to data from a source connector then it will be applied before the hits kafka.  If its for a sink connector then its applied to the data from kafka and before the data hits the sink connector.&lt;/p&gt;

&lt;p&gt;What is the source of the data? Does it support producing data of different formats?  How have you configured the value serializer?&lt;/p&gt;

&lt;p&gt;If you are using the Mongo Kafka Connector as the source and want to apply this SMT then you will have to wait until &lt;a href=&quot;https://jira.mongodb.org/browse/KAFKA-40&quot; title=&quot;Support schema for Source connector&quot; class=&quot;issue-link&quot; data-issue-key=&quot;KAFKA-40&quot;&gt;&lt;del&gt;KAFKA-40&lt;/del&gt;&lt;/a&gt; to support schema&apos;d data from the source.&lt;/p&gt;

&lt;p&gt;I hope that helps,&lt;/p&gt;

&lt;p&gt;Ross&lt;/p&gt;</comment>
                            <comment id="2341048" author="industrial.lang@gmail.com" created="Wed, 24 Jul 2019 09:37:05 +0000"  >&lt;p&gt;Hi Ross, thanks for replay.&lt;/p&gt;

&lt;p&gt;Maybe i don&apos;t fully understand data flow from mongo to kafka through mongo connector.&lt;br/&gt;
 I imagine that process as flow of json documents from mongo to connector and then to kafka. Kafka before add this document to topic can make some transormations(Kafka SMT feature).&lt;br/&gt;
 I try to do some transformations but kafka says &quot;Only Struct objects supported for &lt;span class=&quot;error&quot;&gt;&amp;#91;copying fields from value to key&amp;#93;&lt;/span&gt;, found: java.lang.String&quot;.&lt;/p&gt;

&lt;p&gt;From this I concluded that from the connector to the Kafka not the structure (json) is passed, but the string.&lt;/p&gt;</comment>
                            <comment id="2340420" author="ross@10gen.com" created="Wed, 24 Jul 2019 07:29:33 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=industrial.lang%40gmail.com&quot; class=&quot;user-hover&quot; rel=&quot;industrial.lang@gmail.com&quot;&gt;industrial.lang@gmail.com&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thanks for the ticket.  Just to let you know the value to key mechanism is handled by Kafka connect and not the MongoDB Connector.  From the exception it sounds like your data in the topic is a string and but the value to key mechanism requires it to be a Struct (so Json or Avro).&lt;/p&gt;

&lt;p&gt;Ross&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="841164">KAFKA-40</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hv5uef:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                </customfields>
    </item>
</channel>
</rss>