Uploaded image for project: 'Java Driver'
  1. Java Driver
  2. JAVA-4417

Memory leak in AsynchronousTlsChannelGroup

    • Fully Compatible
    • Not Needed

      We are observing a memory leak related to com.mongodb.internal.connection.tlschannel.async.AsynchronousTlsChannelGroup

      Driver version is 4.3.4 

      Server is Azure CosmosDB using MongoDB Server version 4.0, server-side retries enabled, SSL enabled.

      Application is using spring-boot 2.4.5 and reactive stack.

       

      The leak starts after the following exception is thrown: 

      {{java.nio.channels.ClosedChannelException
      at java.base/java.nio.channels.spi.AbstractSelectableChannel.register(Unknown Source)
      at com.mongodb.internal.connection.tlschannel.async.AsynchronousTlsChannelGroup.registerPendingSockets(AsynchronousTlsChannelGroup.java:612)
      at com.mongodb.internal.connection.tlschannel.async.AsynchronousTlsChannelGroup.loop(AsynchronousTlsChannelGroup.java:414)
      at java.base/java.lang.Thread.run(Unknown Source)}}

      The leak is in com.mongodb.internal.connection.tlschannel.async.AsynchronousTlsChannelGroup#pendingRegistrations

      Pending registrations are accumulating and eventually causing OOM.

      It seems that com.mongodb.connection.TlsChannelStreamFactoryFactory.TlsChannelStream#openAsync will continue to create AsynchronousTlsChannel instances addig new pending registrations even after the com.mongodb.internal.connection.tlschannel.async.AsynchronousTlsChannelGroup#loop was shut down.

      Not exactly clear what is triggering this behavior as we only observed it on one of our environments. It also needs decent time before it happens. Last time it took 2 days before it started.

       

      There are other exceptions thrown prior to this one but hours or days before so not necessarily related:

      java.lang.NullPointerException: Cannot invoke "com.mongodb.internal.connection.tlschannel.impl.BufferHolder.prepare()" because "this.outEncrypted" is null at com.mongodb.internal.connection.tlschannel.impl.TlsChannelImpl.wrapAndWrite(TlsChannelImpl.java:393) at com.mongodb.internal.connection.tlschannel.impl.TlsChannelImpl.write(TlsChannelImpl.java:384) at com.mongodb.internal.connection.tlschannel.ClientTlsChannel.write(ClientTlsChannel.java:184) at com.mongodb.internal.connection.tlschannel.async.AsynchronousTlsChannelGroup.writeHandlingTasks(AsynchronousTlsChannelGroup.java:540) at com.mongodb.internal.connection.tlschannel.async.AsynchronousTlsChannelGroup.doWrite(AsynchronousTlsChannelGroup.java:498) at com.mongodb.internal.connection.tlschannel.async.AsynchronousTlsChannelGroup.lambda$processWrite$4(AsynchronousTlsChannelGroup.java:459) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source)

       

      Another one is: 
      error in operation java.lang.NullPointerException

      Most probably thrown from{{ com.mongodb.internal.connection.tlschannel.async.AsynchronousTlsChannelGroup#processWrite}}

        1. heapdump.png
          59 kB
          Dániel Imre

            Assignee:
            valentin.kovalenko@mongodb.com Valentin Kavalenka
            Reporter:
            d4niel.imre@gmail.com Dániel Imre
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: