[KAFKA-48] Can't set custom message key Created: 23/Jul/19 Updated: 27/Oct/23 Resolved: 25/Jul/19 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Dionis Stepanidis | Assignee: | Ross Lawley |
| Resolution: | Works as Designed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Description |
|
I try to use mongo document id as message key. Following this examples https://docs.confluent.io/current/connect/transforms/valuetokey.html#valuetokey i specify settings in MongoSourceConnector.properties but connector raise exception:
|
| Comments |
| Comment by Ross Lawley [ 25/Jul/19 ] |
|
I'm closing as Works as Designed - only because Kafka and the connector are working as they currently are supposed to. Please follow / watch Ross |
| Comment by Dionis Stepanidis [ 24/Jul/19 ] |
I will wait that feature. Thanks for help Ross |
| Comment by Ross Lawley [ 24/Jul/19 ] |
|
It depends how you are using the SMT feature. If you are applying it to data from a source connector then it will be applied before the hits kafka. If its for a sink connector then its applied to the data from kafka and before the data hits the sink connector. What is the source of the data? Does it support producing data of different formats? How have you configured the value serializer? If you are using the Mongo Kafka Connector as the source and want to apply this SMT then you will have to wait until I hope that helps, Ross |
| Comment by Dionis Stepanidis [ 24/Jul/19 ] |
|
Hi Ross, thanks for replay. Maybe i don't fully understand data flow from mongo to kafka through mongo connector. From this I concluded that from the connector to the Kafka not the structure (json) is passed, but the string. |
| Comment by Ross Lawley [ 24/Jul/19 ] |
|
Thanks for the ticket. Just to let you know the value to key mechanism is handled by Kafka connect and not the MongoDB Connector. From the exception it sounds like your data in the topic is a string and but the value to key mechanism requires it to be a Struct (so Json or Avro). Ross |