Uploaded image for project: 'Kafka Connector'
  1. Kafka Connector
  2. KAFKA-115

Connector stop itself after db timeout.

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Works as Designed
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None

      We are running a single-instance mongo replica-set, sometimes this replica-set is changing availbility zones. It might take a bit of time for the db to be available again when this occurs (typically 2 minutes). 
      During this time the connector loses the connection and shut itself down : 

      com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=mongo:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketException: smatch-hub-mongo}, caused by {java.net.UnknownHostException: smatch-hub-mongo}}]
	at com.mongodb.internal.connection.BaseCluster.getDescription(BaseCluster.java:182)
	at com.mongodb.internal.connection.SingleServerCluster.getDescription(SingleServerCluster.java:41)
	at com.mongodb.client.internal.MongoClientDelegate.getConnectedClusterDescription(MongoClientDelegate.java:152)
	at com.mongodb.client.internal.MongoClientDelegate.createClientSession(MongoClientDelegate.java:103)
	at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.getClientSession(MongoClientDelegate.java:284)
	at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:188)
	at com.mongodb.client.internal.ChangeStreamIterableImpl.execute(ChangeStreamIterableImpl.java:203)
	at com.mongodb.client.internal.ChangeStreamIterableImpl.access$000(ChangeStreamIterableImpl.java:53)
	at com.mongodb.client.internal.ChangeStreamIterableImpl$1.cursor(ChangeStreamIterableImpl.java:128)
	at com.mongodb.client.internal.ChangeStreamIterableImpl$1.iterator(ChangeStreamIterableImpl.java:123)
	at com.mongodb.kafka.connect.source.MongoSourceTask.tryCreateCursor(MongoSourceTask.java:242)
	at com.mongodb.kafka.connect.source.MongoSourceTask.createCursor(MongoSourceTask.java:226)
	at com.mongodb.kafka.connect.source.MongoSourceTask.getNextDocument(MongoSourceTask.java:333)
	at com.mongodb.kafka.connect.source.MongoSourceTask.poll(MongoSourceTask.java:155)
	at org.apache.kafka.connect.runtime.WorkerSourceTask.poll(WorkerSourceTask.java:265)
	at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:232)
	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

      The only way is then to manually restart the connector : 

      Task is being killed and will not recover until manually restarted

      Is it be possible to configure the behaviour of the source connector when encountering com.mongodb.MongoTimeoutException  ?

            Assignee:
            ross@mongodb.com Ross Lawley
            Reporter:
            gilvdm@gmail.com Gil Vander Marcken
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: