[KAFKA-263] Confluent MongoDbAtlasSinkConnector topics.regex Created: 08/Nov/21  Updated: 27/Oct/23  Resolved: 09/Nov/21

Status: Closed
Project: Kafka Connector
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Unknown
Reporter: Denis Collumbien Assignee: Robert Walters
Resolution: Works as Designed Votes: 0
Labels: external-user
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File image-2021-11-08-09-56-11-313.png    
Issue Links:
Related
is related to KAFKA-82 Unable to set "topics.regex" for sink... Closed

 Description   

The correction done in KAFKA-82 is not uploaded to the Confluent Cloud.

Error: CCloud backend error: 1 error occurred:
 * Connector configuration is invalid and contains 2 validation error(s). Errors: topics: org.apache.kafka.common.config.ConfigException: Must configure one of topics or topics.regex,"topics" is required
: topics : [org.apache.kafka.common.config.ConfigException: Must configure one of topics or topics.regex "topics" is required]



 Comments   
Comment by Nathan Nam [ 17/Nov/21 ]

Hello,

I work for Confluent. We have intentionally not exposed ``topics.regex`` due to many reasons including security concerns.

Comment by Denis Collumbien [ 09/Nov/21 ]

Funny, because they mention to on their site to make the ticket here, because you are in control of the connector integration. AND the error is the same error as mentioned in ticket KAFKA-82

I don't like it to be send back and forward... Please open this ticket again and solve the issue. This is not a limitation of the UI as you can set the "special" settings via the the config file.

Comment by Robert Walters [ 09/Nov/21 ]

Confluent Cloud issue not exposing regex parameter

Comment by Robert Walters [ 09/Nov/21 ]

While the Atlas connector is the MongoDB Connector under the covers, Confluent chose to expose a subset of the parameters.  These are entirely under their control and I would recommend reaching out and filing a support ticket with Confluent to add this to their UI.  As a work around you can specify the topic names in a comma separated list or you'll have to stand up a Kafka Connect instance running the MongoDB Connector for Apache Kafka.

Comment by Denis Collumbien [ 09/Nov/21 ]

We are using Confluent Cloud with the default MongoDB Atlas Sink connector that is available here.

The deployment of the connector is done with ccloud command and the json file that you can see in the screenshot.

Comment by Robert Walters [ 08/Nov/21 ]

it@trustbuilder.com Are you using the Confluent Cloud and configuring the MongoDB Atlas Source and Sink?

 

Generated at Thu Feb 08 09:05:58 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.