[KAFKA-166] MongoDB sink connector returns `topic` in validation result when not in ConfigDef or in connector properties Created: 12/Oct/20 Updated: 28/Oct/23 Resolved: 13/Jan/21 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | Sink |
| Affects Version/s: | 1.2.0, 1.3.0 |
| Fix Version/s: | 1.4.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Randall Hauch | Assignee: | Ross Lawley |
| Resolution: | Fixed | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Description |
|
When a new connector configuration for the MongoDB sink connector is validated using Connect, the MongoDB sink connector includes in the validation output the `topic` property (with a value matching the `topics` property) even though no such property is defined in the ConfigDef and is not even included in the connector configuration. For example, validate a connector configuration as follows:
where the mongo-sink.json file contains:
Note that this configuration specified a `topics` (plural) property, but did not specify a `topic` (singular) property. Connect requires either sink connector configurations to specify the topics to be consumed by the connector via a `topics` or `topics.regex` property, but not both. The result of the Connect REST API validation is the following JSON:
AK Connect is currently marking this as an error because it is not defined in the connector's ConfigDef, but that's a bug (https://issues.apache.org/jira/browse/KAFKA-10600). Why does the MongoDB sink connector include in its validation output a `topic` configuration property when it is not required and not even supplied by the user? It's even stranger that the value of this `topic` property matches the value of the user-supplied `topics` property. Perhaps the connector at one time looked for the `topic` property, and keeping that helps maintain backward compatibility. But if this is the only reason, then perhapsĀ this should be transparent to users and the `MongoSinkConnector.validate(...)` method should not include `topic` in its response. |
| Comments |
| Comment by Githook User [ 13/Jan/21 ] |
|
Author: {'name': 'Ross Lawley', 'email': 'ross.lawley@gmail.com', 'username': 'rozza'}Message: Fixed sink validation issue including synthetic config property (#47)
|
| Comment by Ross Lawley [ 12/Jan/21 ] |
| Comment by Randall Hauch [ 12/Oct/20 ] |
|
Consider downgrading the priority for this; it's more of a suggestion to minimize effects on Connect-based tooling. |