[KAFKA-293] record will be lost if offset submitted earlier Created: 16/Feb/22 Updated: 27/Oct/23 Resolved: 15/Mar/22 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Question | Priority: | Major - P3 |
| Reporter: | Liu Zhiqiang | Assignee: | Ross Lawley |
| Resolution: | Works as Designed | Votes: | 0 |
| Labels: | external-user | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
In the class - org.apache.kafka.connect.runtime.WorkerSourceTask firstly, execute but if producer.send(...) fail, but the offset of the record has been submit to offsetWriter and we know that, so it is wrong Is this a bug? |
| Comments |
| Comment by Ross Lawley [ 15/Mar/22 ] |
|
Hi 837500869@qq.com, Thanks for the ticket, the offset mechanism is part of Kafka's resilience and how the Kafka connect framework works with it is outside the remit of the MongoDB Kafka Connector. I think your question is better suited to the Kafka mailing list / development team. All the best, Ross |
| Comment by Liu Zhiqiang [ 25/Feb/22 ] |
|
After I read the code of kafka connect - mongodb source , I have rewrited it by myself , please see https://github.com/lzqdename/pulsar-mongodb-source-connector can everyone give me some improve suggestions? Thank you very much !
|
| Comment by Esha Bhargava [ 22/Feb/22 ] |
|
837500869@qq.com Thank you for reporting this issue! We'll look into it and get back to you soon. |
| Comment by Liu Zhiqiang [ 16/Feb/22 ] |
|
title is wrong, should be "record will be lost if offset submitted earlier" |
| Comment by Liu Zhiqiang [ 16/Feb/22 ] |
|
Affected version-Mongo Kafka Connector 1.6.1 |