[KAFKA-178] MongoSinkConnector nullpointerexception with Confluent Platform 6.0 Created: 01/Dec/20  Updated: 27/Oct/23  Resolved: 21/Dec/20

Status: Closed
Project: Kafka Connector
Component/s: Sink
Affects Version/s: 1.3.0
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: David Wadge Assignee: Ross Lawley
Resolution: Gone away Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Confluent Platform 6.0



 Description   

The MongoDBSinkConnector launched from Confluent Platform 6.0 (https://github.com/confluentinc/cp-all-in-one) immediately produces a nullpointerexception. Downgrading to Confluent Platform 5.5.2 resolves the issue, however we can't do that. 

This is a major blocker for us, is anyone aware of a workaround?

Thanks

 

Config

{
  "connector.class": "com.mongodb.kafka.connect.MongoSinkConnector",
  "name": "mongodb-sink",
  "tasks.max": "1",
  "input.data.format": "JSON",
  "topics":"mytopic",
  "connection.uri": "mongodb+srv://my-user:mypassword@myhost.mongodb.net",
  "connection.host": "myhost.mongodb.net",
  "connection.user": "myuser",
  "connection.password": "mypassword",
  "database": "mydb",
  "delete.on.null.values": "false",
  "document.id.strategy": "com.mongodb.kafka.connect.sink.processor.id.strategy.PartialValueStrategy",
  "value.projection.list":"id",
  "value.projection.type":"AllowList",
  "writemodel.strategy":"com.mongodb.kafka.connect.sink.writemodel.strategy.ReplaceOneBusinessKeyStrategy"
}
 

Stack trace

connect      | [2020-11-30 11:07:57,473] INFO Opened connection [connectionId{localValue:30, serverValue:123679}] to myhost.mongodb.net:27017 (org.mongodb.driver.connection)
connect      | [2020-11-30 11:07:57,654] INFO Closed connection [connectionId{localValue:30, serverValue:123679}] to myhost.mongodb.net:27017 because the pool has been closed. (org.mongodb.driver.connection)
connect      | [2020-11-30 11:07:57,656] ERROR Uncaught exception in REST call to /connectors (org.apache.kafka.connect.runtime.rest.errors.ConnectExceptionMapper)
connect      | java.lang.NullPointerException
connect      |  at org.apache.kafka.connect.runtime.WorkerConfigDecorator$MutableConfigInfos.lambda$removeAllWithName$0(WorkerConfigDecorator.java:295)
connect      |  at org.apache.kafka.connect.runtime.WorkerConfigDecorator$MutableConfigInfos.removeAll(WorkerConfigDecorator.java:305)
connect      |  at org.apache.kafka.connect.runtime.WorkerConfigDecorator$MutableConfigInfos.removeAllWithName(WorkerConfigDecorator.java:294)
connect      |  at org.apache.kafka.connect.runtime.WorkerConfigDecorator$DecorationPattern.filterValidationResults(WorkerConfigDecorator.java:432)
connect      |  at org.apache.kafka.connect.runtime.WorkerConfigDecorator.lambda$decorateValidationResult$5(WorkerConfigDecorator.java:273)
connect      |  at java.base/java.util.Collections$SingletonList.forEach(Collections.java:4856)
connect      |  at org.apache.kafka.connect.runtime.WorkerConfigDecorator.decorateValidationResult(WorkerConfigDecorator.java:273)
connect      |  at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:392)
connect      |  at org.apache.kafka.connect.runtime.AbstractHerder.lambda$validateConnectorConfig$1(AbstractHerder.java:326)
connect      |  at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
connect      |  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
connect      |  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
connect      |  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
connect      |  at java.base/java.lang.Thread.run(Thread.java:834) (edited) 



 Comments   
Comment by Backlog - Core Eng Program Management Team [ 21/Dec/20 ]

There hasn't been any recent activity on this ticket, so we're resolving it. Thanks for reaching out! Please feel free to comment on this if you're able to provide more information.

Comment by Ross Lawley [ 07/Dec/20 ]

Hi dwadge@rheglobal.com,

Thanks for the ticket. We've had a few reports of issues when upgrading to Confluent Platform 6. One user reported that they:

solved the problem by indicating security protocol: “confluent.topic.security.protocol”: “PLAINTEXT”

Can you check if that fixes the issue for you?

Ross

[1] https://developer.mongodb.com/community/forums/t/nullpointer-exception-when-creating-kafka-sink-connector/11596/6

Generated at Thu Feb 08 09:05:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.