[KAFKA-184] Configurable topic suffix for source connector Created: 14/Jan/21  Updated: 22/Jan/21  Resolved: 22/Jan/21

Status: Closed
Project: Kafka Connector
Component/s: Source
Affects Version/s: None
Fix Version/s: 1.4.0

Type: New Feature Priority: Major - P3
Reporter: Mario Molina Assignee: Ross Lawley
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by KAFKA-185 Support dynamic topic naming strategies Closed
Related
related to KAFKA-185 Support dynamic topic naming strategies Closed

 Description   

Support for setting a custom suffix in the topic in which the messages will be sent by the source connector, in the same way as we already have in case of the topic prefix.



 Comments   
Comment by Ross Lawley [ 22/Jan/21 ]

Hi mmolimar@gmail.com,

Thanks for the PR. As part of the 1.4.0 release a new dynamic namespace mapper has been added to the sink. That will allow users to dynamically take SinkRecord data and set the desired namespace for it to be stored in.

KAFKA-185 adds a similar ability to set the topic from the data in the Change Stream document. I will add a SUFFIX configuration to the default topic mapper based on your feedback.

Ross

Comment by Mario Molina [ 14/Jan/21 ]

Thanks Ross.

That works but the idea is to have the same thing as we already have with the topic.prefix: build the topic name with a prefix.db.collection.suffix without using transformations.
There is PR already opened.

Comment by Ross Lawley [ 14/Jan/21 ]

Hi mmolimar@gmail.com,

You can use the RegexRouter to change the topic name from the source. We're in the process of adding an example to the documentation.

Will that meet your needs?

Ross

Generated at Thu Feb 08 09:05:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.