-
Type: Epic
-
Resolution: Fixed
-
Priority: Major - P3
-
Affects Version/s: 1.5.0
-
Component/s: Sink
-
Done
-
Write MongoDB errors to the DLQ
-
8
-
6
-
6
-
100
-
Documentation:
errors.deadletterqueue.topic.name: Name of topic to use as the dead letter queue.
But:
When errors.deadletterqueue.topic.name is a valid Kafka topic name ** And exception happens at com.mongodb.kafka.connect.sink.MongoSinkTask.bulkWriteBatch(MongoSinkTask.java:209) Then no message is send to the DLQ.
Cause probably is in catch block which does not use the ErrantRecordReporter
private void bulkWriteBatch(final List<MongoProcessedSinkRecordData> batch) { ... try { ... } catch (MongoException e) { LOGGER.warn( "Writing {} document(s) into collection [{}] failed.", writeModels.size(), namespace.getFullName()); handleMongoException(config, e); } catch (InterruptedException e) { Thread.currentThread().interrupt(); throw new DataException("Rate limiting was interrupted", e); } catch (Exception e) { if (config.logErrors()) { LOGGER.error("Failed to write mongodb documents", e); } if (!config.tolerateErrors()) { throw new DataException("Failed to write mongodb documents", e); } } }
Developer notes
The scope (WRITING-9407) was rewritten, and this section was moved there.