<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 09:06:13 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[KAFKA-368] MongoDB source connector (&gt;1.9.0) does not release connections</title>
                <link>https://jira.mongodb.org/browse/KAFKA-368</link>
                <project id="16285" key="KAFKA">Kafka Connector</project>
                    <description>&lt;p&gt;Our company faced an issue after migrating to the MongoDB source connector version 1.9.0 (and higher). Description:&lt;br/&gt;
We have an Atlas Cloud MongoDB non-production cluster that is pausing by schedule (at night and at weekends) due to operational costs. Also, we have a self-hosted kafka-connect cluster with an installed MongoDB source connector in there. There&apos;s also a self-hosted application that monitors all the connectors by schedule (i.e. every 30 seconds) and restarts all the failed connectors and tasks. Before the upgrade to 1.9.0, we used version 1.8.1 and everything worked as expected - during the MongoDB downtime (at night) source connector keeps failing and restarted by the app. When the cluster is up again, the number of connections remains roughly the same (see screenshot #1 attached).&lt;/p&gt;

&lt;p&gt;After we migrated to version 1.9.0 (the issue appears also in all the versions up to 1.10.0), the number of connections started to grow each time MongoDB is down at night (see screenshot #2 attached). The connections are kept alive during the time and eventually, it leads to the reaching of the Atlas cluster limits. After that, our connector is not able to consume any changes from MongoDB and write to Kafka.&#160;&lt;br/&gt;
As a quick fix, we had to stop all the connectors and redeploy kafka-connect cluster.&lt;/p&gt;



&lt;p&gt;We&apos;ve tried various MongoDB Java driver configs such as &lt;tt&gt;connectTimeoutM, ocketTimeoutMS, maxIdleTimeMS, maxLifeTimeMS&lt;/tt&gt; but it had no effect on the number of connections.&#160;&lt;/p&gt;

&lt;p&gt;Versions that we use:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Atlas MongoDB cluster - 5.0.17, 6.0&lt;/li&gt;
	&lt;li&gt;Apache Kafka - 3.4.0&lt;/li&gt;
	&lt;li&gt;kafka-connect cluster is built based on JDK 17&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;The only difference between the 2 use cases was MongoDB Source connector (1.8.1 vs 1.9.0).&lt;/p&gt;</description>
                <environment></environment>
        <key id="2335574">KAFKA-368</key>
            <summary>MongoDB source connector (&gt;1.9.0) does not release connections</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="10300" iconUrl="https://jira.mongodb.org/images/icons/priorities/medium.svg">Unknown</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="valentin.kovalenko@mongodb.com">Valentin Kavalenka</assignee>
                                    <reporter username="artsiom.krauchanka@comparethemarket.com">Artsiom Krauchanka</reporter>
                        <labels>
                    </labels>
                <created>Tue, 9 May 2023 10:04:37 +0000</created>
                <updated>Tue, 16 May 2023 07:13:53 +0000</updated>
                            <resolved>Mon, 15 May 2023 17:19:55 +0000</resolved>
                                    <version>1.9.0</version>
                                                                        <votes>0</votes>
                                    <watches>1</watches>
                                                                                                                <comments>
                            <comment id="5427230" author="JIRAUSER1273688" created="Tue, 16 May 2023 07:13:53 +0000"  >&lt;p&gt;Thanks, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=valentin.kovalenko%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;valentin.kovalenko@mongodb.com&quot;&gt;valentin.kovalenko@mongodb.com&lt;/a&gt;, looking forward to the new version 1.10.1!&lt;/p&gt;</comment>
                            <comment id="5425775" author="JIRAUSER1258163" created="Mon, 15 May 2023 17:19:55 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=artsiom.krauchanka%40comparethemarket.com&quot; class=&quot;user-hover&quot; rel=&quot;artsiom.krauchanka@comparethemarket.com&quot;&gt;artsiom.krauchanka@comparethemarket.com&lt;/a&gt;, I believe this is a duplicate of &lt;a href=&quot;https://jira.mongodb.org/browse/KAFKA-370&quot; title=&quot;Fix resource management in MongoSourceTask, MongoSinkTask&quot; class=&quot;issue-link&quot; data-issue-key=&quot;KAFKA-370&quot;&gt;&lt;del&gt;KAFKA-370&lt;/del&gt;&lt;/a&gt;, for which we already have a PR in review. Thank you for reporting the bug!&lt;/p&gt;</comment>
                            <comment id="5410726" author="JIRAUSER1263145" created="Tue, 9 May 2023 10:04:40 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=artsiom.krauchanka%40comparethemarket.com&quot; class=&quot;user-hover&quot; rel=&quot;artsiom.krauchanka@comparethemarket.com&quot;&gt;artsiom.krauchanka@comparethemarket.com&lt;/a&gt;, thank you for reporting this issue! The team will look into it and get back to you soon. &lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="2339742">KAFKA-370</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="448763" name="1. mongodb-source-connector-1.8.1-downtime.png" size="24865" author="artsiom.krauchanka@comparethemarket.com" created="Tue, 9 May 2023 09:51:13 +0000"/>
                            <attachment id="448762" name="2. mongodb-source-connector-1.9.0-downtime.png" size="23749" author="artsiom.krauchanka@comparethemarket.com" created="Tue, 9 May 2023 09:55:30 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10257" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Documentation Changes</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="11861"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i1nimb:xy</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                </customfields>
    </item>
</channel>
</rss>