<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Wed Feb 07 21:09:28 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[CDRIVER-432] Memory leak every ~100 connections</title>
                <link>https://jira.mongodb.org/browse/CDRIVER-432</link>
                <project id="10030" key="CDRIVER">C Driver</project>
                    <description>&lt;p&gt;I have slightly modified the example-client.c, in order to make repeated connections to the mongoDB and i found this two major problems:&lt;/p&gt;

&lt;p&gt;1) a memory leak of 4kb every ~100 connection&lt;/p&gt;


&lt;p&gt;I am very concerned about the problem number 1, because the process begins with a memory usage of about 800kb and after just seconds, get over 2MB!&lt;/p&gt;

&lt;p&gt;In the attached file example-client.c changed.&lt;/p&gt;

&lt;p&gt;Thanks&lt;/p&gt;</description>
                <environment>OS X 10.9.4 and Linux</environment>
        <key id="160662">CDRIVER-432</key>
            <summary>Memory leak every ~100 connections</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="adam.midvidy">Adam Midvidy</assignee>
                                    <reporter username="ech0s7r">ech0s7r</reporter>
                        <labels>
                    </labels>
                <created>Sat, 27 Sep 2014 11:58:13 +0000</created>
                <updated>Mon, 8 Jan 2024 15:30:23 +0000</updated>
                            <resolved>Wed, 1 Oct 2014 18:57:45 +0000</resolved>
                                                    <fixVersion> 1.0.2 </fixVersion>
                                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="731848" author="dgladkikh_splunk" created="Wed, 1 Oct 2014 19:44:47 +0000"  >&lt;p&gt;Adam thank you again for explanation. &lt;/p&gt;</comment>
                            <comment id="731844" author="adam.midvidy" created="Wed, 1 Oct 2014 19:40:13 +0000"  >&lt;p&gt;Yes, with the usleep in place the failure is less deterministic because the OS has additional time to cleanup connection state between connection attempts.&lt;/p&gt;

&lt;p&gt;If you make the sleeps long enough,  the OS will have sufficient time to clean up such that the failures should not happen at all.&lt;/p&gt;

&lt;p&gt;To go into even more detail, after you close the socket the OS will put the socket into a TIME_WAIT state. If the OS were to immediately make the socket available for a new connection there would be a risk of getting packets that were meant for a prior connection, which could cause errors. This is why SO_REUSEADDR is generally a bad idea.&lt;/p&gt;</comment>
                            <comment id="731840" author="adam.midvidy" created="Wed, 1 Oct 2014 19:38:58 +0000"  >&lt;p&gt;No problem, glad to help. You can watch &lt;a href=&quot;https://jira.mongodb.org/browse/CDRIVER-436&quot; title=&quot;Improve error messages for connection failures&quot; class=&quot;issue-link&quot; data-issue-key=&quot;CDRIVER-436&quot;&gt;&lt;del&gt;CDRIVER-436&lt;/del&gt;&lt;/a&gt; if you want to track the improvements to error messages.&lt;/p&gt;</comment>
                            <comment id="731836" author="dgladkikh_splunk" created="Wed, 1 Oct 2014 19:38:23 +0000"  >&lt;p&gt;Adam, actually I have one more question, do you how it is possible that for example after 1000 connection connection #1001 fails and connection #1002 succeeds? Does it mean that OS can do some cleanup work between 1001 and 1002 or driver does that? &lt;/p&gt;</comment>
                            <comment id="731830" author="dgladkikh_splunk" created="Wed, 1 Oct 2014 19:34:29 +0000"  >&lt;p&gt;Adam, thank you for explanation. Just wanted to understand the root cause of this issue. Glad to hear that we should not see this issue if we will keep connections around. &lt;/p&gt;</comment>
                            <comment id="731786" author="adam.midvidy" created="Wed, 1 Oct 2014 19:10:52 +0000"  >&lt;p&gt;The OS still retains some state per-socket after the socket is closed in the client process, so you will still get resource exhaustion regardless of whether the program closes the socket. If you want to check for yourself, remove the usleep call from the example program, recompile , and try rerunning the program until a socket fails, and then immediately rerun it. It should fail immediately the second time.&lt;/p&gt;
</comment>
                            <comment id="731770" author="dgladkikh_splunk" created="Wed, 1 Oct 2014 19:02:50 +0000"  >&lt;p&gt;Adam, but in this example we don&apos;t keep clients alive. We kill them, should mongoc_client_destroy actually close connection/sockets? &lt;/p&gt;</comment>
                            <comment id="731758" author="adam.midvidy" created="Wed, 1 Oct 2014 18:57:36 +0000"  >&lt;p&gt;I also split this into 2 tickets, one for the memory leak, and another for the connection failure. &lt;/p&gt;

&lt;p&gt;See &lt;a href=&quot;https://jira.mongodb.org/browse/CDRIVER-435&quot; title=&quot; Cursor Failure every ~10000 connection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;CDRIVER-435&quot;&gt;&lt;del&gt;CDRIVER-435&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="731754" author="adam.midvidy" created="Wed, 1 Oct 2014 18:53:14 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=dgladkikh_splunk&quot; class=&quot;user-hover&quot; rel=&quot;dgladkikh_splunk&quot;&gt;dgladkikh_splunk&lt;/a&gt; The error you are seeing here is that the operating system is refusing to create so many new sockets to the same host. The driver is capable of many simultaneous connections to different hosts, but the problem here is that they are all to the same host. &lt;/p&gt;

&lt;p&gt;I think &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=ech0s7r&quot; class=&quot;user-hover&quot; rel=&quot;ech0s7r&quot;&gt;ech0s7r&lt;/a&gt; is correct that the real fix here is to reuse the same connection each time you create a cursor. &lt;br/&gt;
I will also file a ticket to improve the error message for this failure condition.&lt;/p&gt;</comment>
                            <comment id="731649" author="ech0s7r" created="Wed, 1 Oct 2014 17:33:02 +0000"  >&lt;p&gt;Denis Gladkikh, yes it fix only problem (1).&lt;br/&gt;
To solve the problem (2), I modified the sample code always keeping open only one connection (create only one client) and execute the 1000000 query. I think the problem still exists and needs to be resolved or probably driver&apos;s engineers doesn&apos;t have expected many connections / disconnections in a short period of time by another process.&lt;/p&gt;</comment>
                            <comment id="731535" author="dgladkikh_splunk" created="Wed, 1 Oct 2014 16:13:48 +0000"  >&lt;p&gt;Adam Midvidy, it seems like that this fix &lt;a href=&quot;https://github.com/mongodb/libbson/pull/92&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/libbson/pull/92&lt;/a&gt; addresses only problem (1), but not a problem (2), am I right? For example I modified a little example provided by @ech0s7r to &lt;a href=&quot;http://pastebin.com/Za194ius&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://pastebin.com/Za194ius&lt;/a&gt; and I still see Cursor Failures. In my case I have seen 50 failures on 50000 iterations. So every 1000 connections.&lt;/p&gt;</comment>
                            <comment id="729468" author="adam.midvidy" created="Mon, 29 Sep 2014 15:43:16 +0000"  >&lt;p&gt;See proposed fix here: &lt;a href=&quot;https://github.com/mongodb/libbson/pull/92&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/libbson/pull/92&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="728786" author="ech0s7r" created="Sat, 27 Sep 2014 14:16:27 +0000"  >&lt;p&gt;I wrote another little test case &lt;a href=&quot;http://pastebin.com/JHKmX49Q&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://pastebin.com/JHKmX49Q&lt;/a&gt; and i have seen that the memory leak occours also only calling mongoc_client_get_collection/mongoc_client_destroy repeatedly. &lt;br/&gt;
The leak was in _mongoc_buffer_destroy() function and I&apos;m fixed with this patch &lt;a href=&quot;http://pastebin.com/YrWFidwR&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://pastebin.com/YrWFidwR&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="161405">CDRIVER-435</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="161406">CDRIVER-436</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="53162" name="example-client.c" size="1943" author="ech0s7r" created="Sat, 27 Sep 2014 11:58:13 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hs2lk7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>139899</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="286">C Sprint 8</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            </customfields>
    </item>
</channel>
</rss>