[JAVA-793] Race condition in com.mongodb.DBTCPConnector#authenticate method Created: 26/Mar/13  Updated: 28/Jul/14  Resolved: 08/Apr/13

Status: Closed
Project: Java Driver
Component/s: Connection Management
Affects Version/s: 2.11.0
Fix Version/s: 2.11.1, 2.12.0

Type: Bug Priority: Major - P3
Reporter: Jeffrey Yemin Assignee: Jeffrey Yemin
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related

 Description   

It's possible for _masterPortPool to be null in the call to MyPort.get from DBTCPConnector.authenticate, causing this mis-leading exception to be thrown:

throw new MongoException("Rare case where master=null, probably all servers are down");

This can be reproduced reliably when constructing a MongoClient with a List of mongos nodes and a call to DB.authenticate is the first operation on the MongoClient (as is typical).

This issue can be worked around by replacing the call to DB.authenticate with a call to the MongoClient constructor that takes a List<MongoCredential>: http://api.mongodb.org/java/current/com/mongodb/MongoClient.html#MongoClient(java.util.List, java.util.List, com.mongodb.MongoClientOptions).



 Comments   
Comment by Jeffrey Yemin [ 27/Mar/13 ]

No problem. We'll likely wait a little while before cutting 2.11.1, just in case any other high priority bugs come through.

Comment by Chris Curtin [ 27/Mar/13 ]

That worked. No more errors while authenticating. Quick query/insert tests also worked okay.

Thanks for the fast turn around!

Comment by Jeffrey Yemin [ 27/Mar/13 ]

Can you test out a new build with a fix for this issue? You can get it here:

https://oss.sonatype.org/content/repositories/snapshots/org/mongodb/mongo-java-driver/2.11.1-SNAPSHOT/mongo-java-driver-2.11.1-20130327.175252-1.jar

Comment by auto [ 27/Mar/13 ]

Author:

{u'date': u'2013-03-27T17:39:12Z', u'name': u'Jeff Yemin', u'email': u'jeff.yemin@10gen.com'}

Message: JAVA-793: Fixed race condition in DBTCPConnector.authenticate by putting in a call to checkMaster prior to getting a DBPort to authenticate on.
Branch: 2.11.x
https://github.com/mongodb/mongo-java-driver/commit/c4a4d7ca7a9c22c4a4e26de04b763f4cdad8b650

Comment by auto [ 27/Mar/13 ]

Author:

{u'date': u'2013-03-27T17:39:12Z', u'name': u'Jeff Yemin', u'email': u'jeff.yemin@10gen.com'}

Message: JAVA-793: Fixed race condition in DBTCPConnector.authenticate by putting in a call to checkMaster prior to getting a DBPort to authenticate on.
Branch: master
https://github.com/mongodb/mongo-java-driver/commit/9c3d8c80feb043a1b565186956548a7cc5de04d1

Comment by Chris Curtin [ 27/Mar/13 ]

Yes, the servers are mongos processes. I've tested with both a single server and a set of servers and the same problem occurs.

Is there a debug level or something I can turn on to get more details?

Comment by Jeffrey Yemin [ 27/Mar/13 ]

Hi Chris,

Just to confirm: the list of servers you are providing are a list of mongos nodes, not a list of replica set members, correct? I want to make sure there's not something wrong with the discovery process.

Comment by Chris Curtin [ 27/Mar/13 ]

First, the error occurs immediately, not after 10 seconds as suggested in the mailing list thread.

Here is the stack of running threads. First is when I successfully auth using the

MongoClient(List<ServerAddress> seeds, List<MongoCredential> credentialsList, MongoClientOptions options) as suggested in the mailing list.

Second is when I use DB.authenticate which fails.

Both cases are thread dumps from IntelliJ when I set a 'sleep' as the last thing in the main() so I could capture the threads.

Success:

MongosStatus:MongosUpdater@361 daemon, prio=5, in group 'main', status: 'SLEEPING'
at java.lang.Thread.sleep(Thread.java:-1)
at com.mongodb.MongosStatus$MongosUpdater.run(MongosStatus.java:79)

MongoCleaner516992923@303 daemon, prio=5, in group 'main', status: 'SLEEPING'
at java.lang.Thread.sleep(Thread.java:-1)
at com.mongodb.Mongo$CursorCleanerThread.run(Mongo.java:773)

Attach Listener@620 daemon, prio=5, in group 'system', status: 'RUNNING'

Signal Dispatcher@621 daemon, prio=9, in group 'system', status: 'RUNNING'

Finalizer@622 daemon, prio=8, in group 'system', status: 'WAIT'
at java.lang.Object.wait(Object.java:-1)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

Reference Handler@623 daemon, prio=10, in group 'system', status: 'WAIT'
at java.lang.Object.wait(Object.java:-1)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)

main@1, prio=5, in group 'main', status: 'RUNNING'
at com.silverpop.mongodb.basic.ConnectionTesting.main(ConnectionTesting.java:45)

Failure:

MongosStatus:MongosUpdater@363 daemon, prio=5, in group 'main', status: 'SLEEPING'
at java.lang.Thread.sleep(Thread.java:-1)
at com.mongodb.MongosStatus$MongosUpdater.run(MongosStatus.java:79)

MongoCleaner1751161119@299 daemon, prio=5, in group 'main', status: 'SLEEPING'
at java.lang.Thread.sleep(Thread.java:-1)
at com.mongodb.Mongo$CursorCleanerThread.run(Mongo.java:773)

Attach Listener@370 daemon, prio=5, in group 'system', status: 'RUNNING'

Signal Dispatcher@371 daemon, prio=9, in group 'system', status: 'RUNNING'

Finalizer@372 daemon, prio=8, in group 'system', status: 'WAIT'
at java.lang.Object.wait(Object.java:-1)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

Reference Handler@373 daemon, prio=10, in group 'system', status: 'WAIT'
at java.lang.Object.wait(Object.java:-1)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)

main@1, prio=5, in group 'main', status: 'RUNNING'
at com.silverpop.mongodb.basic.ConnectionTesting.main(ConnectionTesting.java:45)

Generated at Thu Feb 08 08:53:05 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.