[KAFKA-43] Connector is killed without any information in console Created: 10/Jul/19 Updated: 11/Sep/19 Resolved: 24/Jul/19 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | None |
| Affects Version/s: | 0.1 |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Vu Le | Assignee: | Ross Lawley |
| Resolution: | Cannot Reproduce | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
CentOS Linux release 7.5.1804 |
||
| Description |
|
Hi team, I setup a environment for streaming data from Kafka to MongoDB. My kafka contains more than 4 million messages. I start the Connector at first time. 178337 records are inserted into Mongo successfully. But, then the Connector is killed. The console display (I tried to set log level is TRACE in kafka/config/log4j.properties)
The configuration file MongoSinkConnector.properties
Expectation: All messages from Kafka are inserted into Mongo successfully. |
| Comments |
| Comment by Ross Lawley [ 24/Jul/19 ] |
|
Hi leanhvu1989, I hope you were able to get to the bottom of this. As its not a message from the connector I'm closing as 'cannot reproduce'. I also found a great blog post about Kafka connect logging which may help. Ross |
| Comment by Vu Le [ 10/Jul/19 ] |
|
Hi Ross, Thanks for your response. I check kafka/logs folder, but there is no log of Kafka Connect (I only see server.log and controller.log). How to enable logging for Kafka Connect? It seems that the Kafka node has problem. Maybe, it results in the connector to be stopped after streaming a number of messages. I also double-check with another node (200k messages). However, I got problem what I raised KAFKA-42 Thank you. |
| Comment by Ross Lawley [ 10/Jul/19 ] |
|
Hi leanhvu1989, Thanks for the ticket - the Killed logging message didn't come from the Mongo Kafka connector codebase. So its unclear as to the cause of the connector being Killed. Is there any logging from Kafka Connect? Could the node have been killed or the connector itself been stopped? Ross |