[JAVA-3754] Stray monitor thread left behind if authentication fails Created: 04/Jun/20  Updated: 27/Oct/23  Resolved: 08/Jun/20

Status: Closed
Project: Java Driver
Component/s: Connection Management
Affects Version/s: 3.8.2, 3.12.5
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Gaurav Bahrani Assignee: Jeffrey Yemin
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

MacOS


Attachments: PNG File Screenshot 2020-06-04 at 7.49.59 AM.png    

 Description   

If incorrect credentials are supplied while setting up connection to replica-set, it results in stray monitor thread left behind. Since 'mongoClient' never got set, caller can not call close() to force driver to release resources.

------
MongoCredential credential = MongoCredential.createCredential(dbUser, authDB, dbPassword.toCharArray()); // credentials supplied are incorrect

ServerAddress address = new ServerAddress(host, localPort);

MongoClientOptions options =
MongoClientOptions.builder()
.serverSelectionTimeout(serverSelectionTimeout)
.connectTimeout(connecTimeout)
.readPreference(ReadPreference.secondaryPreferred())
.sslEnabled(sslEnabled)
.sslInvalidHostNameAllowed(true)
.build();

try

{ MongoClient mongoClient = new MongoClient(address, credential, options); }

catch (MongoSecurityException mse)

{ // handle login error }

catch (MongoTimeoutException mte)

{ // handle timeout }

catch (Exception ex)

{ // handle other errors }

 Comments   
Comment by Gaurav Bahrani [ 09/Jun/20 ]

@jeff.yemin@mongodb.com

Figured out the issue. It seems to be happening with 3.8.2 driver. When I upgraded to 3.12.5 version, the issue went away.

Thanks for the help in diagnosing the issue.

Comment by Jeffrey Yemin [ 08/Jun/20 ]

gaurav-hevo

Unfortunately, I am not able to reproduce it my own testing.  With a 10 second sleep after the call to close, the only threads left are:

 
Thread[Finalizer,8,system]
Thread[Common-Cleaner,8,InnocuousThreadGroup]
Thread[main,5,main]
Thread[Reference Handler,10,system]
Thread[Signal Dispatcher,9,system]
Thread[Notification Thread,9,system

 

So currently we do not have a way to proceed unless you can provide more information.

Comment by Gaurav Bahrani [ 08/Jun/20 ]

@jeffery, shall we reopen this ticket?

Comment by Gaurav Bahrani [ 08/Jun/20 ]

Even with 5mins sleep stray thread keeps hanging around.

Comment by Gaurav Bahrani [ 08/Jun/20 ]

I was suspecting the same, hence tried adding 2mins sleep, but that also doesn't help. In my application, we create Mongo connections on need basis and overtime we are seeing 100s of stray threads lying around.

Comment by Jeffrey Yemin [ 08/Jun/20 ]

Try adding a Thread.sleep() after the MongoClient.close().  Some of the threads are closed asynchronously using Thread.interrupt(), so they may not have had a chance to be cleaned up by the time you check.

Comment by Gaurav Bahrani [ 08/Jun/20 ]

@jeffrey here's the code

import com.mongodb.*;
 
import java.util.Iterator;
import java.util.Set;
 
public class MongoConnectionTest {
    public static void main(String[] args) {
        MongoClientOptions options =
                MongoClientOptions.builder()
                        .serverSelectionTimeout(30000)
                        .connectTimeout(30000)
                        .readPreference(ReadPreference.secondaryPreferred())
                        .sslEnabled(true)
                        .sslInvalidHostNameAllowed(true)
                        .build();
 
        ServerAddress address = new ServerAddress("xxxx.mongodb.net", 27017);
 
        MongoCredential credential = MongoCredential.createCredential("admin", "admin", "xxxx".toCharArray());
 
        MongoClient mongoClient = new MongoClient(address, credential, options);
 
        try {
            System.out.println("DB List");
 
            Iterator<String> dbList = mongoClient.listDatabaseNames().iterator();
 
            while (dbList.hasNext()) {
                System.out.println(dbList.next());
            }
        } catch (RuntimeException rxe) {
            System.out.println(rxe);
        } finally {
            if (mongoClient != null) {
                try {
                    mongoClient.close();
                } catch (RuntimeException rxe) {
                    System.out.println(rxe);
                }
            }
 
            Set<Thread> threadSet = Thread.getAllStackTraces().keySet();
 
            System.out.println("Threads list");
            for (Thread th: threadSet) {
                System.out.println(th.toString());
            }
        }
    }
}

For successful run, here is the print

DB List
test
admin
local
Threads list
Thread[Reference Handler,10,system]
Thread[Attach Listener,9,system]
Thread[Monitor Ctrl-Break,5,main]
Thread[Finalizer,8,system]
Thread[main,5,main]
Thread[Signal Dispatcher,9,system]

For failure run (bad user/password), here is the print

DB List
com.mongodb.MongoSecurityException: Exception authenticating MongoCredential{mechanism=SCRAM-SHA-1, userName='admn', source='admin', password=<hidden>, mechanismProperties={}}
com.mongodb.MongoSecurityException: Exception authenticating MongoCredential{mechanism=SCRAM-SHA-1, userName='admn', source='admin', password=<hidden>, mechanismProperties={}}
Threads list
Thread[Signal Dispatcher,9,system]
Thread[CleanCursors-1-thread-1,5,main]
Thread[Attach Listener,9,system]
Thread[cluster-ClusterId{value='5ede3f35449efb5e30fe9bcc', description='null'}-xxxx.mongodb.net:27017,5,main] <-- Stray thread
Thread[Finalizer,8,system]
Thread[Monitor Ctrl-Break,5,main]
Thread[Reference Handler,10,system]
Thread[main,5,main]

As you can see in the failure scenario, there is a "cluster-ClusterId" hanging around.

Comment by Jeffrey Yemin [ 08/Jun/20 ]

gaurav-hevo the initial connections themselves all happen in background threads, so the constructor will return normally and not throw an exception. Try it, and I think you'll see that what I'm saying is correct. If I'm wrong, please provide a full program and log output demonstrating the issue that you're seeing.

Comment by Gaurav Bahrani [ 08/Jun/20 ]

@jeffrey, if I instantiate MongoClient using empty argument constructor, then it will try to connect to a Mongo locally (localhost, 27017). I want to connect to a remote Mongo server, so I must pass host, port and credentials. As per documentation, there isn't a connect/open method which I could use to explicitly connect to a remote Mongo after constructing the client.

Comment by Jeffrey Yemin [ 08/Jun/20 ]

The MongoClient constructor won't throw any exceptions after starting any threads. So you should be able to do:

MongoClient client = new MongoClient();
try { 
   ...
} finally {
   client.close();
}

and not leak any threads.

Generated at Thu Feb 08 09:00:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.