<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 08:34:12 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[GODRIVER-431] Memory leak in InsertMany?</title>
                <link>https://jira.mongodb.org/browse/GODRIVER-431</link>
                <project id="14289" key="GODRIVER">Go Driver</project>
                    <description>&lt;p&gt;Is it possible that there is a memory leak when InsertMany is used?&lt;/p&gt;

&lt;p&gt;I created a simple program &lt;a href=&quot;https://gist.github.com/thomasgeulen/3968d31baf225d4873f74e789b61e3b5&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;GIST&lt;/a&gt;&#160;and get the following output in pprof&lt;/p&gt;

&lt;p&gt;Showing nodes accounting for 22425.05kB, 100% of 22425.05kB total&lt;br/&gt;
Showing top 10 nodes out of 23&lt;br/&gt;
 flat flat% sum% cum cum%&lt;br/&gt;
16144.01kB 71.99% 71.99% 22425.05kB 100% main.main.func2&lt;br/&gt;
 3638.89kB 16.23% 88.22% 3638.89kB 16.23% github.com/mongodb/mongo-go-driver/core/wiremessage.Query.AppendWireMessage&lt;br/&gt;
 1536.07kB 6.85% 95.07% 1536.07kB 6.85% github.com/mongodb/mongo-go-driver/bson.newElement&lt;br/&gt;
 553.04kB 2.47% 97.53% 553.04kB 2.47% github.com/mongodb/mongo-go-driver/bson.NewDocument&lt;br/&gt;
 553.04kB 2.47% 100% 6281.04kB 28.01% github.com/mongodb/mongo-go-driver/mongo.(*Collection).InsertMany&lt;br/&gt;
 0 0% 100% 553.04kB 2.47% github.com/mongodb/mongo-go-driver/bson.ElementConstructor.ArrayFromElements&lt;br/&gt;
 0 0% 100% 512.02kB 2.28% github.com/mongodb/mongo-go-driver/bson.ElementConstructor.ObjectID&lt;br/&gt;
 0 0% 100% 1024.05kB 4.57% github.com/mongodb/mongo-go-driver/bson.ElementConstructor.SubDocument&lt;br/&gt;
 0 0% 100% 553.04kB 2.47% github.com/mongodb/mongo-go-driver/bson.NewArray&lt;br/&gt;
 0 0% 100% 1024.05kB 4.57% github.com/mongodb/mongo-go-driver/bson.ValueConstructor.Document&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;When I use the same simple program and add every struct one by one with InsertOne, I cannot see that something is waiting to get garbage collected.&lt;/p&gt;</description>
                <environment></environment>
        <key id="549872">GODRIVER-431</key>
            <summary>Memory leak in InsertMany?</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="kris.brandow@mongodb.com">Kristofer Brandow</assignee>
                                    <reporter username="thomas.geulen">Thomas Geulen</reporter>
                        <labels>
                    </labels>
                <created>Fri, 25 May 2018 17:51:31 +0000</created>
                <updated>Wed, 11 Sep 2019 18:35:00 +0000</updated>
                            <resolved>Thu, 31 May 2018 16:38:01 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="1902541" author="kris.brandow" created="Sun, 27 May 2018 00:16:37 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=thomas.geulen&quot; class=&quot;user-hover&quot; rel=&quot;thomas.geulen&quot;&gt;thomas.geulen&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;TL;DR: This isn&apos;t a memory leak, it&apos;s an underlying optimization.&lt;/p&gt;

&lt;p&gt;On the connections we create we make a read and write buffer to lower the amount of pressure we put on the garbage collector and the number of allocations we need to do. When we create these buffers, we allocate a slice with a 256 byte capacity. The total size of the wire protocol message for the documents in the example is 116 bytes. This means that the AppendWireMessage method will not allocate a new slice when used with &lt;tt&gt;InsertOne&lt;/tt&gt;. When &lt;tt&gt;InsertMany&lt;/tt&gt;&#160;is used the batch size is much larger than 256 bytes, so AppendWireMessage creates a new slice. Since the &lt;tt&gt;mongo.Client&lt;/tt&gt;&#160;created for the HTTP request is not &lt;tt&gt;Disconnect}}ed after being used, the connection remains in the pool indefinitely. Even when the {{mongo.Client&lt;/tt&gt; is garbage collected, the underlying &lt;tt&gt;topology.Topology&lt;/tt&gt; still has goroutines running which means that everything below the &lt;tt&gt;topology.Topology&lt;/tt&gt; including the &lt;tt&gt;connection.Pool&lt;/tt&gt; cannot be garbage collected. This keeps the connection used alive, along with it&apos;s not increased write buffer. This is what you&apos;re seeing. I believe pprof shows where something was allocated, which is why &lt;tt&gt;wiremessage.Query.AppendWireMessage&lt;/tt&gt; is shown there.&lt;/p&gt;

&lt;p&gt;Let me know if this clears things up.&lt;/p&gt;

&lt;p&gt;As a bonus, I did discover that we are leaking timers while debugging this. So while the exact thing you filed a bug for wasn&apos;t a memory leak, there was a memory leak related to this code.&lt;/p&gt;

&lt;p&gt;--Kris&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="550274">GODRIVER-432</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htq1i7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            </customfields>
    </item>
</channel>
</rss>