[JAVA-2053] Default NioEventLoopGroup created by driver must be closed Created: 04/Dec/15  Updated: 27/Feb/19  Resolved: 04/Oct/16

Status: Closed
Project: Java Driver
Component/s: Async
Affects Version/s: 3.2.0
Fix Version/s: 3.4.0

Type: Improvement Priority: Major - P3
Reporter: Jeffrey Yemin Assignee: Jeffrey Yemin
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related

 Description   

From https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/mongodb-user/ynz9LfmgjV0/igj2MXgXAwAJ

We're using the async driver in a webapp deployed to Tomcat (v8, along with Java 8). After Tomcat is shut down, its log file (catalina.out) has several entries that read something like this:

The web application appears to have started a thread named nioEventLoopGroup-2-1 but has failed to stop it. This is very likely to create a memory leak.

The culprit is com.mongodb.connection.netty.NettyStreamFactoryFactory. It creates an instance of NioEventLoopGroup but never shuts it down.

We're using Spring and our solution was to write our own StreamFactoryFactory that is identical to NettyStreamFactoryFactory except in two ways. It's annotated with @Component (so its lifecycle is managed by Spring) and it has the following method:

@PreDestroy
public void shutdown()
{
  eventLoopGroup.shutdownGracefully().awaitUniterruptibly();
}

For those not using Spring I recommend creating doing something like this...

EventLoopGroup myEventLoopGroup = new NioEventLoopGroup();
StreamFactoryFactory myStreamFactoryFactory = new NettyStreamFactoryFactory(myEventLoopGroup, ByteBufAllocator.DEFAULT);

...and shut down myEventLoopGroup yourself at the appropriate time.

The async driver should properly manage the EventLoopGroup it creates...we had to kill Tomcat manually until we figured out the simple fix.



 Comments   
Comment by Jeffrey Yemin [ 27/Feb/19 ]

Also, just so you're aware the driver now has "native" async TLS support so Netty is no longer strictly required.

I think it's going to be difficult for the driver to handle this seamlessly given that at the point the driver is given the factory it's as a StreamFactoryFactory, which doesn't even have a close method, and even if it did, the driver wouldn't know whether the instance is being shared in the application with multiple MongoClient instances.

I guess the question is whether it's worth adding a resource closer callback to MongoClientSettings just to handle this case, or leave it up to the application to close the EventLoopGroup after closing the MongoClient.

Comment by Sean Fitts [ 27/Feb/19 ]

Fair enough – I did manage to miss that.  Is there a way to get the external resource closer set that I missed as well (the only path I saw for that involved using the connection string, which I don't want to do.

Comment by Sean Fitts [ 27/Feb/19 ]

Also, even in the case where the event loop group is supplied by the caller, it doesn't seem possible to get the cleanup to happen when you would like.  Ideally I'd like to pass in the handler to call when things are being shut down (aka have some way to set the external resource closer), but there is no way to do that currently.

Comment by Jeffrey Yemin [ 27/Feb/19 ]

Seems we had to do that for backwards compatibility. We did add this Javadoc, but I can see how it's easy to miss.

        /**
         * Sets the event loop group.
         *
         * <p>It is highly recommended to supply your own event loop group and manage its shutdown.  Otherwise, the event
         * loop group created by default will not be shutdown properly.</p>
         *
         * @param eventLoopGroup the event loop group that all channels created by this factory will be a part of
         * @return this
         */
        public Builder eventLoopGroup(final EventLoopGroup eventLoopGroup) {
            this.eventLoopGroup = notNull("eventLoopGroup", eventLoopGroup);
            return this;
        }

Comment by Sean Fitts [ 27/Feb/19 ]

Actually in the purely default case where you call "NettyStreamFactoryFactory.builder().build()" the event loop group is created by code in the driver (line 153 of NettyStreamFactoryFactory.java).  So there really isn't any indication that the caller needs to do lifecycle management of what appears to be an internal implementation detail of the default factory.  It would seem that in this case the factory should also have a way to install the necessary clean up handler.

Comment by Jeffrey Yemin [ 27/Feb/19 ]

Yes, that's true. Since the application is now supplying the EventLoopGroup (to NettyStreamFactoryFactory.Builder#eventLoopGroup}}, then the application is responsible for its life cycle and must decide when to close it.

Let me know if I'm misunderstanding you.

Comment by Sean Fitts [ 27/Feb/19 ]

With the most recent async/reactive streams driver (3.10.x) this appears to have resurfaced.  If you use the non-deprecated mechanisms for setting the stream factory (MongoClientSettings.Builder.streamFactoryFactory) and you set it to use the NettyStreamFactoryFactory, then the code added to fix this bug is not executed.  In this case the requestedStreamType is null in com.mongodb.async.client.MonogClients.create (the private version) and so the "externalResourceCloser" that was added to close the eventloop group is never set.

As a result you can still leak event groups.

Comment by Githook User [ 04/Oct/16 ]

Author:

{u'username': u'jyemin', u'name': u'Jeff Yemin', u'email': u'jeff.yemin@10gen.com'}

Message: JAVA-2053: When no stream factory factory is provided, ensure that any EventLoopGroup instances are shutdown cleanly
Branch: master
https://github.com/mongodb/mongo-java-driver/commit/56a1ae0a134c1a25614b4303c4fd67b40a219d8a

Generated at Thu Feb 08 08:56:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.