[KAFKA-42] how to handle non-JSON format messages when streaming data from Kafka to Mongo Created: 10/Jul/19 Updated: 11/Sep/19 Resolved: 10/Jul/19 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | None |
| Affects Version/s: | 0.1 |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Vu Le | Assignee: | Ross Lawley |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Windows 7, CentOS Linux release 7.5.1804 |
||
| Attachments: |
|
| Description |
|
Hi team, I can stream data from Kafka to MongoDB with JSON messages However, if I send a non-JSON format message the Connector died. Please see the log file for details. My config file:
I have 2 separated questions:
Thanks |
| Comments |
| Comment by Vu Le [ 10/Jul/19 ] |
|
Hi Ross, Thank for your response. I raised this on stackoverflow already. But, so far, there is no answer. I will reach to Kafka mailing list as you mentioned. Thank you again. Vu Le |
| Comment by Ross Lawley [ 10/Jul/19 ] |
|
Hi leanhvu1989, Thanks for the ticket. It seems its Kafka connect that is throwing the error before anything is handled by the MongoDB Sink Connector. As such there is nothing the connector itself can do. Just for future reference the best place for questions regarding MongoDB usage or Kafka connector specifics is the mongodb-user mailinglist or stackoverflow as you will reach a broader audience there. If your business requires an answer from MongoDB within a time frame then we do offer production support. In this case I think using Kafka's own mailing list may be the be best way to get insight into how best to handle this scenario. All the best, Ross |