[KAFKA-254] Allow Source Connector to Automatically Restart After "Resume Token Not Found" Created: 05/Oct/21 Updated: 28/Oct/23 Resolved: 18/Oct/21 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | Source |
| Affects Version/s: | None |
| Fix Version/s: | 1.3.0 |
| Type: | Improvement | Priority: | Unknown |
| Reporter: | Diego Rodriguez (Inactive) | Assignee: | Ross Lawley |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | internal-user | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||
| Case: | (copied to CRM) | ||||
| Documentation Changes: | Needed | ||||
| Documentation Changes Summary: | If implemented, this new behavior will need to be properly documented. |
||||
| Description |
|
Hi Team, Right now, when the Source Connector encounters a "Resume Token Not Found" error it fails and doesn't automatically recover. The reasoning behind this decision is that there are data loss implications when this happens and the safe approach is to stop, allow the user to understand the implications, and carefully decide the next steps. Would it be possible to make this more flexible by adding a configuration option that allows the Source Connector to automatically restart from scratch whenever a "Resume Token Not Found" error is found? A clear message that this happened and the implications should be written to the logs, as well as a startup banner that states the currently configured behavior and its implications. Thanks |
| Comments |
| Comment by Robert Walters [ 18/Oct/21 ] |
|
Fixed as part of error tolerance work in 1.3 |
| Comment by Ross Lawley [ 11/Oct/21 ] |
|
Hi diego.rodriguez, With added support for errors.tolerance=all in Does that setting meet the need for the user? Ross |