<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 08:56:52 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[JAVA-2302] Memory leak on Java async driver + Netty 4.1.x</title>
                <link>https://jira.mongodb.org/browse/JAVA-2302</link>
                <project id="10006" key="JAVA">Java Driver</project>
                    <description>&lt;p&gt;We&apos;ve been using Netty as the MongoDB transport in our Java application and it&apos;s been working fine.&lt;br/&gt;
However, since upgrading to Netty 4.1, we began running into memory leaks.&lt;br/&gt;
The errors look like the following: &lt;/p&gt;

&lt;p&gt;   com.mongodb.MongoException: failed to allocate 16777216 byte(s) of direct memory (used: 4596957223, max: 4608491520)&lt;/p&gt;

&lt;p&gt;It starts happening after doing lots of Mongo queries.&lt;br/&gt;
By changing the transport back to the default NIO2, the leak is gone and everything works again.&lt;/p&gt;

&lt;p&gt;We hope we can go back to using Netty though, as it performs better.&lt;/p&gt;

&lt;p&gt;The issue happened with both Netty 4.1.1 and 4.1.4. It seems related to Netty 4.1.x releases in general.&lt;br/&gt;
We&apos;re using the async driver 3.3.0 and RX driver 1.2.0 (though RX is probably unrelated to the problem).&lt;/p&gt;</description>
                <environment>Java 8, Vert.x 3.3.2, CentOS Linux release 7.2.1511 (Core)</environment>
        <key id="315108">JAVA-2302</key>
            <summary>Memory leak on Java async driver + Netty 4.1.x</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="ross@mongodb.com">Ross Lawley</assignee>
                                    <reporter username="fschuh">Frederico Ferro Schuh</reporter>
                        <labels>
                    </labels>
                <created>Fri, 9 Sep 2016 08:39:08 +0000</created>
                <updated>Wed, 19 Oct 2016 14:17:43 +0000</updated>
                            <resolved>Mon, 19 Sep 2016 19:11:54 +0000</resolved>
                                    <version>3.3.0</version>
                                    <fixVersion>3.4.0-rc1</fixVersion>
                    <fixVersion>3.4.0</fixVersion>
                                    <component>Async</component>
                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="1388308" author="fschuh" created="Tue, 20 Sep 2016 02:39:48 +0000"  >&lt;p&gt;Great news Ross.&lt;br/&gt;
We&apos;ll be testing your fix under heavy load and I&apos;ll report back here if we detect any further leaks.&lt;/p&gt;</comment>
                            <comment id="1387277" author="xgen-internal-githook" created="Mon, 19 Sep 2016 08:45:54 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;rozza&apos;, u&apos;name&apos;: u&apos;Ross Lawley&apos;, u&apos;email&apos;: u&apos;ross.lawley@gmail.com&apos;}
&lt;p&gt;Message: Netty fixes&lt;/p&gt;

&lt;p&gt;Update Netty to 4.1.5.final&lt;br/&gt;
Fix leak in NettyStream&lt;br/&gt;
Made connection string streamType the authorative setting for the streamFactoryFactory&lt;br/&gt;
Added deprecation warning in NettyByteBuf&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/JAVA-2302&quot; title=&quot;Memory leak on Java async driver + Netty 4.1.x&quot; class=&quot;issue-link&quot; data-issue-key=&quot;JAVA-2302&quot;&gt;&lt;del&gt;JAVA-2302&lt;/del&gt;&lt;/a&gt;&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo-java-driver/commit/b952e419d955fec927bb16b8f1fba2e680ff6a66&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo-java-driver/commit/b952e419d955fec927bb16b8f1fba2e680ff6a66&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1386154" author="ross@10gen.com" created="Fri, 16 Sep 2016 13:10:37 +0000"  >&lt;p&gt;Thanks &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=fschuh&quot; class=&quot;user-hover&quot; rel=&quot;fschuh&quot;&gt;fschuh&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Your test program was instrumental in identifying a leak.  Once the leak was fixed I was no longer able to hit your issue, I ran the test to 100k iterations.&lt;/p&gt;

&lt;p&gt;Ross&lt;/p&gt;</comment>
                            <comment id="1386151" author="ross@10gen.com" created="Fri, 16 Sep 2016 13:04:59 +0000"  >&lt;p&gt;PR: &lt;a href=&quot;https://github.com/rozza/mongo-java-driver/pull/162&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/rozza/mongo-java-driver/pull/162&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1386044" author="fschuh" created="Fri, 16 Sep 2016 07:08:55 +0000"  >&lt;p&gt;Hello Ross,&lt;/p&gt;

&lt;p&gt;sorry for not posting the whole stack trace before.&lt;br/&gt;
I&apos;ve came up with a test case that reproduces the issue.&lt;br/&gt;
It&apos;s just a simple test that runs an upsert and a query (with index creation) in an infinite loop.&lt;/p&gt;

&lt;p&gt;Please set the max heap size as small as you can. &lt;br/&gt;
With a 50 mb heap you can get the error to show very quickly (it happened after about 16000 iterations here, on Linux CentOS 7).&lt;/p&gt;

&lt;p&gt;Tested with Netty 4.1.1 Final.&lt;/p&gt;

&lt;p&gt;Heres what the stack trace looks like:&lt;/p&gt;

&lt;p&gt;Exception in thread &quot;main&quot; com.mongodb.MongoInternalException: Unexpected exception&lt;br/&gt;
        at com.mongodb.connection.InternalStreamConnection.translateReadException(InternalStreamConnection.java:487)&lt;br/&gt;
        at com.mongodb.connection.InternalStreamConnection.access$1900(InternalStreamConnection.java:66)&lt;br/&gt;
        at com.mongodb.connection.InternalStreamConnection$3.failed(InternalStreamConnection.java:451)&lt;br/&gt;
        at com.mongodb.connection.netty.NettyStream.readAsync(NettyStream.java:221)&lt;br/&gt;
        at com.mongodb.connection.netty.NettyStream.handleReadResponse(NettyStream.java:255)&lt;br/&gt;
        at com.mongodb.connection.netty.NettyStream.access$800(NettyStream.java:62)&lt;br/&gt;
        at com.mongodb.connection.netty.NettyStream$InboundBufferHandler.exceptionCaught(NettyStream.java:314)&lt;br/&gt;
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:278)&lt;br/&gt;
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:257)&lt;br/&gt;
        at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:249)&lt;br/&gt;
        at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)&lt;br/&gt;
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:278)&lt;br/&gt;
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:257)&lt;br/&gt;
        at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:249)&lt;br/&gt;
        at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1281)&lt;br/&gt;
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:278)&lt;br/&gt;
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:257)&lt;br/&gt;
        at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:893)&lt;br/&gt;
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:93)&lt;br/&gt;
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134)&lt;br/&gt;
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:563)&lt;br/&gt;
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:504)&lt;br/&gt;
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:418)&lt;br/&gt;
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:390)&lt;br/&gt;
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:742)&lt;br/&gt;
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:145)&lt;br/&gt;
        at java.lang.Thread.run(Thread.java:745)&lt;br/&gt;
Caused by: io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 8192 byte(s) of direct memory (used: 50723840, max: 50724864)&lt;br/&gt;
        at io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:592)&lt;br/&gt;
        at io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:546)&lt;br/&gt;
        at io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)&lt;br/&gt;
        at io.netty.buffer.UnpooledUnsafeDirectByteBuf.&amp;lt;init&amp;gt;(UnpooledUnsafeDirectByteBuf.java:68)&lt;br/&gt;
        at io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.&amp;lt;init&amp;gt;(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)&lt;br/&gt;
        at io.netty.buffer.UnsafeByteBufUtil.newUnsafeDirectByteBuf(UnsafeByteBufUtil.java:625)&lt;br/&gt;
        at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:265)&lt;br/&gt;
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)&lt;br/&gt;
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:170)&lt;br/&gt;
        at io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:131)&lt;br/&gt;
        at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:73)&lt;br/&gt;
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:111)&lt;br/&gt;
        ... 7 more&lt;/p&gt;</comment>
                            <comment id="1385352" author="ross@10gen.com" created="Thu, 15 Sep 2016 15:25:16 +0000"  >&lt;p&gt;Hi,&lt;/p&gt;

&lt;p&gt;I&apos;ve been unable to reproduce this issue.  Do you have a test case that causes it?  Also a stacktrace might throw more light on the cause.&lt;/p&gt;

&lt;p&gt;Ross&lt;/p&gt;</comment>
                            <comment id="1380358" author="fschuh" created="Fri, 9 Sep 2016 11:52:40 +0000"  >&lt;p&gt;Thanks for the quick response Ross.&lt;br/&gt;
Let me know if you need more info on my side.&lt;/p&gt;</comment>
                            <comment id="1380325" author="ross@10gen.com" created="Fri, 9 Sep 2016 09:24:09 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=fschuh&quot; class=&quot;user-hover&quot; rel=&quot;fschuh&quot;&gt;fschuh&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thanks for the ticket, we currently test against netty 4.0.x. From what you describe it seems there is some behavioural changes in the 4.1.x series. I&apos;ll investigate and see if we can trigger the same error and determine a fix.&lt;/p&gt;

&lt;p&gt;Ross&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="274178">JAVA-2157</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="138752" name="TestMongoNettyLeak.java" size="3790" author="fschuh" created="Fri, 16 Sep 2016 07:08:54 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hspm5j:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="1200">JVM Sprint 35</customfieldvalue>
    <customfieldvalue id="1302">JVM Sprint 36</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            </customfields>
    </item>
</channel>
</rss>