[KAFKA-81] Schema registry credentials not taken from kafka connect Created: 30/Jan/20 Updated: 17/Jun/20 Resolved: 17/Jun/20 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | None |
| Affects Version/s: | 1.0 |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Ludovic Dussart | Assignee: | Ross Lawley |
| Resolution: | Cannot Reproduce | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
kafka connect 5.3.1 |
||
| Description |
|
hi,
We are using Mongo Sink with the schema registry. The schema registry authentication is handle by the kafka connect instance thanks to the related keys :
When deploying a mongo-sink configuration, if we not add thoses keys, connect throws an Unauthorized exception when trying to get key and value schemas from registry. With others connectors like sftp connect (provided by Confluent), this configuration is induced from kafka-connect configuration listed below.
It will be nice that the Mongo connector hava the same behavior. |
| Comments |
| Comment by Ross Lawley [ 05/Feb/20 ] |
|
Hi ludovic.dussart.pro@gmail.com, Thanks for the clarification. This is strange and looks to me to be a Kafka Connect bug. As mentioned previously the MongoDB Connector does nothing with any configurations outside its remit. From the documentation you've linked it looks like the workers configuration should be used and if specified overrides the connector configurations. So from that it should work and as it doesn't this sounds like a Kafka Connect issue. Ross |
| Comment by Ludovic Dussart [ 04/Feb/20 ] |
|
hi @Ross,
According to the Kafka Connect documentation : Confluent Platform first looks for converter configurations in the connector. If none are found there, the settings in the Connect worker configs are used. Therefore, you have options as to where to specify these properties, which affects how they are inherited among the worker and connectors.
The two last items was apparently not managed by the Mongo Connector because i need to explicitly declared value|key.converter.basic.auth.credentials.source and value|key.converter.basic.auth.user.info even if my kafka connect worker have those configurations |
| Comment by Ross Lawley [ 04/Feb/20 ] |
|
Hi ludovic.dussart.pro@gmail.com, Thanks for the ticket, I'm not sure I follow I think the example config was missing from the description. The MongoDB Connector itself does not handle or use the (key|value.converter) namespace, rather it is purely used by the connect framework. See the Using Kafka Connect with Schema Registry documentation for more information about how to configure a connector with the schema registry. All the best, Ross |