[KAFKA-368] MongoDB source connector (>1.9.0) does not release connections Created: 09/May/23  Updated: 16/May/23  Resolved: 15/May/23

Status: Closed
Project: Kafka Connector
Component/s: None
Affects Version/s: 1.9.0
Fix Version/s: None

Type: Bug Priority: Unknown
Reporter: Artsiom Krauchanka Assignee: Valentin Kavalenka
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File 1. mongodb-source-connector-1.8.1-downtime.png     PNG File 2. mongodb-source-connector-1.9.0-downtime.png    
Issue Links:
Duplicate
duplicates KAFKA-370 Fix resource management in MongoSourc... Closed
Documentation Changes: Not Needed

 Description   

Our company faced an issue after migrating to the MongoDB source connector version 1.9.0 (and higher). Description:
We have an Atlas Cloud MongoDB non-production cluster that is pausing by schedule (at night and at weekends) due to operational costs. Also, we have a self-hosted kafka-connect cluster with an installed MongoDB source connector in there. There's also a self-hosted application that monitors all the connectors by schedule (i.e. every 30 seconds) and restarts all the failed connectors and tasks. Before the upgrade to 1.9.0, we used version 1.8.1 and everything worked as expected - during the MongoDB downtime (at night) source connector keeps failing and restarted by the app. When the cluster is up again, the number of connections remains roughly the same (see screenshot #1 attached).

After we migrated to version 1.9.0 (the issue appears also in all the versions up to 1.10.0), the number of connections started to grow each time MongoDB is down at night (see screenshot #2 attached). The connections are kept alive during the time and eventually, it leads to the reaching of the Atlas cluster limits. After that, our connector is not able to consume any changes from MongoDB and write to Kafka. 
As a quick fix, we had to stop all the connectors and redeploy kafka-connect cluster.

We've tried various MongoDB Java driver configs such as connectTimeoutM, ocketTimeoutMS, maxIdleTimeMS, maxLifeTimeMS but it had no effect on the number of connections. 

Versions that we use:

  • Atlas MongoDB cluster - 5.0.17, 6.0
  • Apache Kafka - 3.4.0
  • kafka-connect cluster is built based on JDK 17

The only difference between the 2 use cases was MongoDB Source connector (1.8.1 vs 1.9.0).



 Comments   
Comment by Artsiom Krauchanka [ 16/May/23 ]

Thanks, valentin.kovalenko@mongodb.com, looking forward to the new version 1.10.1!

Comment by Valentin Kavalenka [ 15/May/23 ]

Hi artsiom.krauchanka@comparethemarket.com, I believe this is a duplicate of KAFKA-370, for which we already have a PR in review. Thank you for reporting the bug!

Comment by Service Account: DBX TPM [ 09/May/23 ]

Hi artsiom.krauchanka@comparethemarket.com, thank you for reporting this issue! The team will look into it and get back to you soon.

Generated at Thu Feb 08 09:06:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.