[KAFKA-138] Improve resume change stream error message Created: 06/Aug/20 Updated: 27/Oct/23 Resolved: 17/Aug/20 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Andrey B | Assignee: | Ross Lawley |
| Resolution: | Works as Designed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
In https://jira.mongodb.org/browse/KAFKA-91 added error message. Kafka Connect uses its own mechanism, not kafka consumer group to store offsets for source connectors. So, as far as I understand there is no way to use kafka-consumer-groups.sh to reset offsets. If I'm wrong about using kafka-consumer-groups.sh, please make the message more explicit. Could be helpful: |
| Comments |
| Comment by Andrey B [ 17/Aug/20 ] |
|
Hi, my mistake. I wasn't clear enough. I mean Source connector usesĀ OffsetStorageReader and OffsetStorageWriter to work with offsets. So, there is no consumer group and kafka-consumer-groups.sh can't be used. |
| Comment by Ross Lawley [ 17/Aug/20 ] |
|
Offsets are handled by Kafka outside of the connector. The connector itself just declares an offset when producing SourceRecords. This allows for changes in Kafka topologies and connector restarts. As far as I'm aware the Kafka connect uses the existing Kafka offset mechanisms and as such you can either delete the offset file when running standalone or manage the meta data in the offset topic via the mentioned shell script or manually. As |