Details
-
Improvement
-
Resolution: Done
-
Major - P3
-
None
-
3.0.2
-
None
Description
I'm trying to test a fail-over scenario with MongoDB Java Client:
- connect to replica set with 3 nodes (1 primary + 2 secondary)
- kill the primary
- new primary node is elected
- (expected) driver continues to work after failing over
Actual:
- MongoDB java client tries to Rediscovering type of existing primary, gets Exception in monitor thread while connecting to server nosql-x64-node-2:27017 and gets stuck:
20:19:52,352 INFO cluster:71 - Cluster created with settings {hosts=[nosql-x64-node-1:27017], mode=MULTIPLE, requiredClusterType=REPLICA_SET, serverSelectionTimeout='30000 ms', maxWaitQueueSize=500, requiredReplicaSetName='rs1'}20:19:52,353 INFO cluster:71 - Adding discovered server nosql-x64-node-1:27017 to client view of cluster20:19:52,483 INFO cluster:71 - No server chosen by PrimaryServerSelector from cluster description ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE, all=[ServerDescription{address=nosql-x64-node-1:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out20:19:52,487 INFO connection:71 - Opened connection [connectionId{localValue:1, serverValue:125}] to nosql-x64-node-1:2701720:19:52,493 INFO cluster:71 - Monitor thread successfully connected to server with description ServerDescription{address=nosql-x64-node-1:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 0, 2]}, minWireVersion=0, maxWireVersion=3, electionId=null, maxDocumentSize=16777216, roundTripTimeNanos=1508516, setName='rs1', canonicalAddress=nosql-x64-node-1:27017, hosts=[nosql-x64-node-3:27017, nosql-x64-node-2:27017, nosql-x64-node-1:27017], passives=[], arbiters=[], primary='nosql-x64-node-2:27017', tagSet=TagSet{[]}}20:19:52,496 INFO cluster:71 - Adding discovered server nosql-x64-node-3:27017 to client view of cluster20:19:52,499 INFO cluster:71 - Adding discovered server nosql-x64-node-2:27017 to client view of cluster20:19:52,505 INFO connection:71 - Opened connection [connectionId{localValue:2, serverValue:169}] to nosql-x64-node-3:2701720:19:52,507 INFO connection:71 - Opened connection [connectionId{localValue:3, serverValue:161}] to nosql-x64-node-2:2701720:19:52,508 INFO cluster:71 - Monitor thread successfully connected to server with description ServerDescription{address=nosql-x64-node-3:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 0, 2]}, minWireVersion=0, maxWireVersion=3, electionId=null, maxDocumentSize=16777216, roundTripTimeNanos=1761189, setName='rs1', canonicalAddress=nosql-x64-node-3:27017, hosts=[nosql-x64-node-3:27017, nosql-x64-node-2:27017, nosql-x64-node-1:27017], passives=[], arbiters=[], primary='nosql-x64-node-2:27017', tagSet=TagSet{[]}}20:19:52,510 INFO cluster:71 - Monitor thread successfully connected to server with description ServerDescription{address=nosql-x64-node-2:27017, type=REPLICA_SET_PRIMARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 0, 2]}, minWireVersion=0, maxWireVersion=3, electionId=55844af54ac7e7537155bd46, maxDocumentSize=16777216, roundTripTimeNanos=1413436, setName='rs1', canonicalAddress=nosql-x64-node-2:27017, hosts=[nosql-x64-node-3:27017, nosql-x64-node-2:27017, nosql-x64-node-1:27017], passives=[], arbiters=[], primary='nosql-x64-node-2:27017', tagSet=TagSet{[]}}20:19:52,510 INFO cluster:71 - Discovered replica set primary nosql-x64-node-2:2701720:19:52,518 INFO connection:71 - Opened connection [connectionId{localValue:4, serverValue:162}] to nosql-x64-node-2:27017...20:21:12,521 INFO cluster:71 - Monitor thread successfully connected to server with description ServerDescription{address=nosql-x64-node-3:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 0, 2]}, minWireVersion=0, maxWireVersion=3, electionId=55844f8645d8c9dd9bbdebfe, maxDocumentSize=16777216, roundTripTimeNanos=1103740, setName='rs1', canonicalAddress=nosql-x64-node-3:27017, hosts=[nosql-x64-node-3:27017, nosql-x64-node-2:27017, nosql-x64-node-1:27017], passives=[], arbiters=[], primary='nosql-x64-node-3:27017', tagSet=TagSet{[]}}20:21:13,013 INFO cluster:71 - Monitor thread successfully connected to server with description ServerDescription{address=nosql-x64-node-1:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 0, 2]}, minWireVersion=0, maxWireVersion=3, electionId=null, maxDocumentSize=16777216, roundTripTimeNanos=1120967, setName='rs1', canonicalAddress=nosql-x64-node-1:27017, hosts=[nosql-x64-node-3:27017, nosql-x64-node-2:27017, nosql-x64-node-1:27017], passives=[], arbiters=[], primary='nosql-x64-node-3:27017', tagSet=TagSet{[]}}20:21:32,525 INFO cluster:71 - Monitor thread successfully connected to server with description ServerDescription{address=nosql-x64-node-3:27017, type=REPLICA_SET_PRIMARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 0, 2]}, minWireVersion=0, maxWireVersion=3, electionId=55844f8645d8c9dd9bbdebfe, maxDocumentSize=16777216, roundTripTimeNanos=1063299, setName='rs1', canonicalAddress=nosql-x64-node-3:27017, hosts=[nosql-x64-node-3:27017, nosql-x64-node-2:27017, nosql-x64-node-1:27017], passives=[], arbiters=[], primary='nosql-x64-node-3:27017', tagSet=TagSet{[]}}20:21:32,526 INFO cluster:71 - Discovered replica set primary nosql-x64-node-3:2701720:21:32,527 INFO cluster:71 - Rediscovering type of existing primary nosql-x64-node-2:2701720:21:34,814 INFO cluster:76 - Exception in monitor thread while connecting to server nosql-x64-node-2:27017com.mongodb.MongoSocketOpenException: Exception opening socketat com.mongodb.connection.SocketStream.open(SocketStream.java:63)at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:114)at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:127)at java.lang.Thread.run(Thread.java:745)Caused by: java.net.NoRouteToHostException: No route to hostat java.net.PlainSocketImpl.socketConnect(Native Method)at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)at java.net.Socket.connect(Socket.java:579)at com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:50)at com.mongodb.connection.SocketStream.open(SocketStream.java:58)... 3 more