[KAFKA-46] Review Sink Connector Scalability Created: 22/Jul/19 Updated: 21/Apr/20 Resolved: 21/Apr/20 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Epic | Priority: | Major - P3 |
| Reporter: | Scott L'Hommedieu (Inactive) | Assignee: | Unassigned |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | sp-toscope | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
to allow more than one worker while maintaining message delivery guarantees (no dups, in order). |
| Comments |
| Comment by Esha Bhargava [ 21/Apr/20 ] |
|
Dup of |
| Comment by Ross Lawley [ 24/Feb/20 ] |
|
martin.andersson@kambi.com just to clarify the Sink connector only supports a single task to handle the writes. The purpose of this ticket is to determine if multiple worker tasks can be used to increase throughput, eg. one task per topic. An alternative at this time would be to register multiple connectors eg one per topic and that would have the same resulting effect. Ross |
| Comment by Martin Andersson [ 29/Jan/20 ] |
|
It strikes me as a bit odd that the connector is not still in Beta when it seems there are still issues like this open. As a user it leaves me wondering if the connector is at all stable and/or functions with all the necessary guarantees. |