[KAFKA-46] Review Sink Connector Scalability Created: 22/Jul/19  Updated: 21/Apr/20  Resolved: 21/Apr/20

Status: Closed
Project: Kafka Connector
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Epic Priority: Major - P3
Reporter: Scott L'Hommedieu (Inactive) Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: sp-toscope
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

to allow more than one worker while maintaining message delivery guarantees (no dups, in order).



 Comments   
Comment by Esha Bhargava [ 21/Apr/20 ]

Dup of KAFKA-92

Comment by Ross Lawley [ 24/Feb/20 ]

martin.andersson@kambi.com just to clarify the Sink connector only supports a single task to handle the writes.

The purpose of this ticket is to determine if multiple worker tasks can be used to increase throughput, eg. one task per topic. An alternative at this time would be to register multiple connectors eg one per topic and that would have the same resulting effect.

Ross

Comment by Martin Andersson [ 29/Jan/20 ]

It strikes me as a bit odd that the connector is not still in Beta when it seems there are still issues like this open. 

As a user it leaves me wondering if the connector is at all stable and/or functions with all the necessary guarantees. 

Generated at Thu Feb 08 09:05:26 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.