[JAVA-2839] Memory leaks when using NettyStream and CommandListener Created: 23/Apr/18  Updated: 28/Oct/23  Resolved: 17/Sep/18

Status: Closed
Project: Java Driver
Component/s: Async
Affects Version/s: 3.6.3
Fix Version/s: 3.8.2, 3.9.0

Type: Bug Priority: Major - P3
Reporter: koji lin Assignee: Ross Lawley
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Netty 4.1.22


Issue Links:
Related
related to JAVA-2460 Memory leaks when using NettyStream a... Closed

 Description   

Met same issue as https://jira.mongodb.org/browse/JAVA-2460
Looks like the problem was fixed for
https://github.com/mongodb/mongo-java-driver/blob/0aa2ec20d5215c0ac727602dd2cd891c22c69ba8/driver-core/src/main/com/mongodb/connection/ByteBufBsonDocument.java#L59-L92

But when we use command listener, I found our code will use ByteBufBsonDocument#createList at
https://github.com/mongodb/mongo-java-driver/blob/0aa2ec20d5215c0ac727602dd2cd891c22c69ba8/driver-core/src/main/com/mongodb/connection/ByteBufBsonDocument.java#L43
and maybe duplicate here needs to be release?
https://github.com/mongodb/mongo-java-driver/blob/0aa2ec20d5215c0ac727602dd2cd891c22c69ba8/driver-core/src/main/com/mongodb/connection/ByteBufBsonDocument.java#L51



 Comments   
Comment by Githook User [ 17/Sep/18 ]

Author:

{'name': 'Ross Lawley', 'email': 'ross.lawley@gmail.com', 'username': 'rozza'}

Message: Ensure ByteBufBsonDocument.createList releases ByteBufs

JAVA-2839
Branch: 3.8.x
https://github.com/mongodb/mongo-java-driver/commit/a62621df20ff0a0752a366a2bbd3179131ed9883

Comment by Githook User [ 17/Sep/18 ]

Author:

{'name': 'Ross Lawley', 'email': 'ross.lawley@gmail.com', 'username': 'rozza'}

Message: Ensure ByteBufBsonDocument.createList releases ByteBufs

JAVA-2839
Branch: master
https://github.com/mongodb/mongo-java-driver/commit/d383be520db90cda828202043852df0570de595a

Comment by Jeffrey Yemin [ 21/Aug/18 ]

I ran the above test program with -Dio.netty.leakDetection.level=paranoid (note the name change to the system property) against a 3.0 mongod server, and have detected leaks

14:39:53.129 [nioEventLoopGroup-2-3] ERROR io.netty.util.ResourceLeakDetector - LEAK: ByteBuf.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
WARNING: 117 leak records were discarded because the leak record count is limited to 4. Use system property io.netty.leakDetection.maxRecords to increase the limit.
Recent access records: 4
#4:
	io.netty.buffer.AdvancedLeakAwareByteBuf.getInt(AdvancedLeakAwareByteBuf.java:186)
	io.netty.buffer.CompositeByteBuf._getInt(CompositeByteBuf.java:807)
	io.netty.buffer.AbstractByteBuf.readInt(AbstractByteBuf.java:767)
	io.netty.buffer.SwappedByteBuf.readInt(SwappedByteBuf.java:581)
	io.netty.buffer.WrappedByteBuf.readInt(WrappedByteBuf.java:571)
	io.netty.buffer.AdvancedLeakAwareByteBuf.readInt(AdvancedLeakAwareByteBuf.java:433)
	com.mongodb.connection.netty.NettyByteBuf.getInt(NettyByteBuf.java:179)
	com.mongodb.connection.ByteBufBsonDocument.createList(ByteBufBsonDocument.java:49)
	com.mongodb.connection.QueryProtocol.asFindCommandResponseDocument(QueryProtocol.java:479)
	com.mongodb.connection.QueryProtocol.sendQuerySucceededEvent(QueryProtocol.java:362)
	com.mongodb.connection.QueryProtocol.access$200(QueryProtocol.java:55)
	com.mongodb.connection.QueryProtocol$QueryResultCallback.callCallback(QueryProtocol.java:528)
	com.mongodb.connection.ResponseCallback.onResult(ResponseCallback.java:48)
	com.mongodb.connection.ResponseCallback.onResult(ResponseCallback.java:23)
	com.mongodb.connection.DefaultConnectionPool$PooledConnection$3.onResult(DefaultConnectionPool.java:497)
	com.mongodb.connection.DefaultConnectionPool$PooledConnection$3.onResult(DefaultConnectionPool.java:491)
	com.mongodb.connection.UsageTrackingInternalConnection$4.onResult(UsageTrackingInternalConnection.java:141)
	com.mongodb.connection.UsageTrackingInternalConnection$4.onResult(UsageTrackingInternalConnection.java:137)
	com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:49)
	com.mongodb.connection.InternalStreamConnection$4.onResult(InternalStreamConnection.java:462)
	com.mongodb.connection.InternalStreamConnection$4.onResult(InternalStreamConnection.java:455)
	com.mongodb.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:635)
	com.mongodb.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:602)
	com.mongodb.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:478)
	com.mongodb.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:475)
	com.mongodb.connection.netty.NettyStream.readAsync(NettyStream.java:236)
	com.mongodb.connection.netty.NettyStream.handleReadResponse(NettyStream.java:266)
	com.mongodb.connection.netty.NettyStream.access$600(NettyStream.java:66)
	com.mongodb.connection.netty.NettyStream$InboundBufferHandler.channelRead0(NettyStream.java:325)
	com.mongodb.connection.netty.NettyStream$InboundBufferHandler.channelRead0(NettyStream.java:322)
	io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341)
	io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349)
	io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
	io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
	io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:642)
	io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:565)
	io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:479)
	io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:441)
	io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	java.base/java.lang.Thread.run(Thread.java:844)
#3:
	io.netty.buffer.AdvancedLeakAwareByteBuf.getByte(AdvancedLeakAwareByteBuf.java:150)
	io.netty.buffer.CompositeByteBuf._getByte(CompositeByteBuf.java:752)
	io.netty.buffer.UnpooledDuplicatedByteBuf._getByte(UnpooledDuplicatedByteBuf.java:34)
	io.netty.buffer.AbstractByteBuf.readByte(AbstractByteBuf.java:689)
	io.netty.buffer.SwappedByteBuf.readByte(SwappedByteBuf.java:531)
	io.netty.buffer.WrappedByteBuf.readByte(WrappedByteBuf.java:521)
	io.netty.buffer.AdvancedLeakAwareByteBuf.readByte(AdvancedLeakAwareByteBuf.java:397)
	com.mongodb.connection.netty.NettyByteBuf.get(NettyByteBuf.java:125)
	org.bson.io.ByteBufferBsonInput.readByte(ByteBufferBsonInput.java:72)
	org.bson.BsonBinaryReader.readBsonType(BsonBinaryReader.java:92)
	org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:149)
	org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:45)
	com.mongodb.connection.ReplyMessage.<init>(ReplyMessage.java:51)
	com.mongodb.connection.QueryProtocol$QueryResultCallback.callCallback(QueryProtocol.java:526)
	com.mongodb.connection.ResponseCallback.onResult(ResponseCallback.java:48)
	com.mongodb.connection.ResponseCallback.onResult(ResponseCallback.java:23)
	com.mongodb.connection.DefaultConnectionPool$PooledConnection$3.onResult(DefaultConnectionPool.java:497)
	com.mongodb.connection.DefaultConnectionPool$PooledConnection$3.onResult(DefaultConnectionPool.java:491)
	com.mongodb.connection.UsageTrackingInternalConnection$4.onResult(UsageTrackingInternalConnection.java:141)
	com.mongodb.connection.UsageTrackingInternalConnection$4.onResult(UsageTrackingInternalConnection.java:137)
	com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:49)
	com.mongodb.connection.InternalStreamConnection$4.onResult(InternalStreamConnection.java:462)
	com.mongodb.connection.InternalStreamConnection$4.onResult(InternalStreamConnection.java:455)
	com.mongodb.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:635)
	com.mongodb.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:602)
	com.mongodb.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:478)
	com.mongodb.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:475)
	com.mongodb.connection.netty.NettyStream.readAsync(NettyStream.java:236)
	com.mongodb.connection.netty.NettyStream.handleReadResponse(NettyStream.java:266)
	com.mongodb.connection.netty.NettyStream.access$600(NettyStream.java:66)
	com.mongodb.connection.netty.NettyStream$InboundBufferHandler.channelRead0(NettyStream.java:325)
	com.mongodb.connection.netty.NettyStream$InboundBufferHandler.channelRead0(NettyStream.java:322)
	io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341)
	io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349)
	io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
	io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
	io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:642)
	io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:565)
	io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:479)
	io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:441)
	io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	java.base/java.lang.Thread.run(Thread.java:844)
#2:
	io.netty.buffer.AdvancedLeakAwareByteBuf.getBytes(AdvancedLeakAwareByteBuf.java:246)
	io.netty.buffer.CompositeByteBuf.getBytes(CompositeByteBuf.java:864)
	io.netty.buffer.CompositeByteBuf.getBytes(CompositeByteBuf.java:44)
	io.netty.buffer.DuplicatedByteBuf.getBytes(DuplicatedByteBuf.java:226)
	io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:851)
	io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:858)
	io.netty.buffer.SwappedByteBuf.readBytes(SwappedByteBuf.java:659)
	io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:649)
	io.netty.buffer.AdvancedLeakAwareByteBuf.readBytes(AdvancedLeakAwareByteBuf.java:493)
	com.mongodb.connection.netty.NettyByteBuf.get(NettyByteBuf.java:135)
	org.bson.io.ByteBufferBsonInput.readBytes(ByteBufferBsonInput.java:79)
	org.bson.BsonBinaryReader.doReadBinaryData(BsonBinaryReader.java:147)
	org.bson.AbstractBsonReader.readBinaryData(AbstractBsonReader.java:283)
	org.bson.codecs.BinaryCodec.decode(BinaryCodec.java:37)
	org.bson.codecs.BinaryCodec.decode(BinaryCodec.java:29)
	org.bson.codecs.DocumentCodec.readValue(DocumentCodec.java:222)
	org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:151)
	org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:45)
	com.mongodb.connection.ReplyMessage.<init>(ReplyMessage.java:51)
	com.mongodb.connection.QueryProtocol$QueryResultCallback.callCallback(QueryProtocol.java:526)
	com.mongodb.connection.ResponseCallback.onResult(ResponseCallback.java:48)
	com.mongodb.connection.ResponseCallback.onResult(ResponseCallback.java:23)
	com.mongodb.connection.DefaultConnectionPool$PooledConnection$3.onResult(DefaultConnectionPool.java:497)
	com.mongodb.connection.DefaultConnectionPool$PooledConnection$3.onResult(DefaultConnectionPool.java:491)
	com.mongodb.connection.UsageTrackingInternalConnection$4.onResult(UsageTrackingInternalConnection.java:141)
	com.mongodb.connection.UsageTrackingInternalConnection$4.onResult(UsageTrackingInternalConnection.java:137)
	com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:49)
	com.mongodb.connection.InternalStreamConnection$4.onResult(InternalStreamConnection.java:462)
	com.mongodb.connection.InternalStreamConnection$4.onResult(InternalStreamConnection.java:455)
	com.mongodb.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:635)
	com.mongodb.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:602)
	com.mongodb.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:478)
	com.mongodb.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:475)
	com.mongodb.connection.netty.NettyStream.readAsync(NettyStream.java:236)
	com.mongodb.connection.netty.NettyStream.handleReadResponse(NettyStream.java:266)
	com.mongodb.connection.netty.NettyStream.access$600(NettyStream.java:66)
	com.mongodb.connection.netty.NettyStream$InboundBufferHandler.channelRead0(NettyStream.java:325)
	com.mongodb.connection.netty.NettyStream$InboundBufferHandler.channelRead0(NettyStream.java:322)
	io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341)
	io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349)
	io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
	io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
	io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:642)
	io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:565)
	io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:479)
	io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:441)
	io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	java.base/java.lang.Thread.run(Thread.java:844)
#1:
	io.netty.buffer.AdvancedLeakAwareByteBuf.getByte(AdvancedLeakAwareByteBuf.java:150)
	io.netty.buffer.CompositeByteBuf._getByte(CompositeByteBuf.java:752)
	io.netty.buffer.UnpooledDuplicatedByteBuf._getByte(UnpooledDuplicatedByteBuf.java:34)
	io.netty.buffer.AbstractByteBuf.readByte(AbstractByteBuf.java:689)
	io.netty.buffer.SwappedByteBuf.readByte(SwappedByteBuf.java:531)
	io.netty.buffer.WrappedByteBuf.readByte(WrappedByteBuf.java:521)
	io.netty.buffer.AdvancedLeakAwareByteBuf.readByte(AdvancedLeakAwareByteBuf.java:397)
	com.mongodb.connection.netty.NettyByteBuf.get(NettyByteBuf.java:125)
	org.bson.io.ByteBufferBsonInput.readByte(ByteBufferBsonInput.java:72)
	org.bson.BsonBinaryReader.doReadBinaryData(BsonBinaryReader.java:137)
	org.bson.AbstractBsonReader.readBinaryData(AbstractBsonReader.java:283)
	org.bson.codecs.BinaryCodec.decode(BinaryCodec.java:37)
	org.bson.codecs.BinaryCodec.decode(BinaryCodec.java:29)
	org.bson.codecs.DocumentCodec.readValue(DocumentCodec.java:222)
	org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:151)
	org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:45)
	com.mongodb.connection.ReplyMessage.<init>(ReplyMessage.java:51)
	com.mongodb.connection.QueryProtocol$QueryResultCallback.callCallback(QueryProtocol.java:526)
	com.mongodb.connection.ResponseCallback.onResult(ResponseCallback.java:48)
	com.mongodb.connection.ResponseCallback.onResult(ResponseCallback.java:23)
	com.mongodb.connection.DefaultConnectionPool$PooledConnection$3.onResult(DefaultConnectionPool.java:497)
	com.mongodb.connection.DefaultConnectionPool$PooledConnection$3.onResult(DefaultConnectionPool.java:491)
	com.mongodb.connection.UsageTrackingInternalConnection$4.onResult(UsageTrackingInternalConnection.java:141)
	com.mongodb.connection.UsageTrackingInternalConnection$4.onResult(UsageTrackingInternalConnection.java:137)
	com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:49)
	com.mongodb.connection.InternalStreamConnection$4.onResult(InternalStreamConnection.java:462)
	com.mongodb.connection.InternalStreamConnection$4.onResult(InternalStreamConnection.java:455)
	com.mongodb.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:635)
	com.mongodb.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:602)
	com.mongodb.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:478)
	com.mongodb.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:475)
	com.mongodb.connection.netty.NettyStream.readAsync(NettyStream.java:236)
	com.mongodb.connection.netty.NettyStream.handleReadResponse(NettyStream.java:266)
	com.mongodb.connection.netty.NettyStream.access$600(NettyStream.java:66)
	com.mongodb.connection.netty.NettyStream$InboundBufferHandler.channelRead0(NettyStream.java:325)
	com.mongodb.connection.netty.NettyStream$InboundBufferHandler.channelRead0(NettyStream.java:322)
	io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341)
	io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349)
	io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
	io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
	io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:642)
	io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:565)
	io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:479)
	io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:441)
	io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	java.base/java.lang.Thread.run(Thread.java:844)
Created at:
	io.netty.util.ResourceLeakDetector.track(ResourceLeakDetector.java:219)
	io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:296)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:170)
	io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:131)
	io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:73)
	io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117)
	io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:642)
	io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:565)
	io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:479)
	io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:441)
	io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	java.base/java.lang.Thread.run(Thread.java:844)

Comment by koji lin [ 21/Jun/18 ]

I forget to provide my mongodb version, looks like only <3.2 version like 3.1, 3.0 will enter that method.

Comment by Jeffrey Yemin [ 20/Jun/18 ]

kojilin I tried to reproduce the failure you're reporting, with the following test program:

import com.mongodb.MongoClientSettings;
import com.mongodb.async.client.MongoClients;
import com.mongodb.connection.netty.NettyStreamFactoryFactory;
import com.mongodb.event.CommandFailedEvent;
import com.mongodb.event.CommandListener;
import com.mongodb.event.CommandStartedEvent;
import com.mongodb.event.CommandSucceededEvent;
import org.bson.Document;
 
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CountDownLatch;
 
import static java.util.concurrent.TimeUnit.SECONDS;
 
public class JAVA2460 {
    public static void main(String[] args) throws InterruptedException {
        var client = MongoClients.create(
                MongoClientSettings.builder()
                        .addCommandListener(new CommandListener() {
                            @Override
                            public void commandStarted(final CommandStartedEvent event) {
                                System.out.println(event.getCommandName());
                            }
 
                            @Override
                            public void commandSucceeded(final CommandSucceededEvent event) {
                                System.out.println(event.getCommandName() + " done");
                                System.out.println();
                            }
 
                            @Override
                            public void commandFailed(final CommandFailedEvent event) {
                                System.out.println(event.getCommandName() + "failed: " + event.getThrowable());
                                System.out.println();
                            }
                        })
                        .streamFactoryFactory(NettyStreamFactoryFactory.builder().build())
                        .build()
        );
 
        var collection = client.getDatabase("test").getCollection("JAVA2460");
 
        List<Document> documents = new ArrayList<>();
        for (int i = 0; i < 1000; i++) {
            documents.add(new Document("b", new byte[1024]));
        }
 
        while (true) {
            var insertLatch = new CountDownLatch(1);
            collection.insertMany(documents, (result, t) -> insertLatch.countDown());
            insertLatch.await(5, SECONDS);
 
            var findLatch = new CountDownLatch(1);
            collection.find().into(new ArrayList<>(), (result, t) -> findLatch.countDown());
            findLatch.await(5, SECONDS);
 
            var dropLatch = new CountDownLatch(1);
            collection.drop((result, t) -> dropLatch.countDown());
            dropLatch.await(5, SECONDS);
        }
    }
}

Running with -Dio.netty.leakDetectionLevel=advanced against the 3.7.1 driver, I don't see any log messages from Netty indicating a leak.

Can you provide more details about how you were detecting the failure that you're reporting here so that we can reproduce it?

Generated at Thu Feb 08 08:58:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.