[CSHARP-1295] Requiring all seed processes to be working in initial connection defeats the purpose of HA of a replication set Created: 22/May/15  Updated: 28/May/15  Resolved: 27/May/15

Status: Closed
Project: C# Driver
Component/s: Connectivity
Affects Version/s: 2.0
Fix Version/s: None

Type: Bug Priority: Blocker - P1
Reporter: tryshi Assignee: Craig Wilson
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive logs.zip    

 Description   

Problem:

In initial connection, if one of the seed mongo processes is down, the connection won't be established. a machine/process can be down for a long time, this means no new connection can be established during this time, this defeats the purpose of High Availability of a replication set.

Expected Result:

As long as one of the seed process is working, connection should be established.

Note:

Not repro in NodeJS driver. so it's C# driver's issue.

Error:

at MongoDB.Driver.Core.Servers.ClusterableServer.<HeartbeatAsync>d__d.MoveNext()" }, { ServerId: "

{ ClusterId : 1, EndPoint : "Unspecified/vmpri0521e.westus.cloudapp.azure.com:27000" }

", EndPoint: "Unspecified/vmpri0521e.westus.cloudapp.azure.com:27000", State: "Disconnected", Type: "Unknown", HeartbeatException: "MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server. ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 191.236.92.13:27000



 Comments   
Comment by Craig Wilson [ 27/May/15 ]

Great.

I'm going to close this as Works as Designed. As we discuss internally what to do regarding whitespace, I'll link back to this ticket with a new one if we are going to do that.

Thanks for your persistence. I know this took a little while. Feel free to delete some of the comments if you wish the public not to have access to your instances.
Craig

Comment by tryshi [ 27/May/15 ]

You're absolutely right, after removing the extra space, everything works beautifully, bad dns in seed and killing primary won't affect the availability.
Thanks a million!

From MongoDB manual ( http://docs.mongodb.org/manual/reference/connection-string/ ), the format does not have space. But as a programmer, I sometimes add extra space here and there for readability. so it will be great we either fix the driver or make it clear in the manual that extra space is not allowed.

Again, thank a lot for your time and effort. Mongodb does provide excellent user experience and support, I really love it!

Comment by Craig Wilson [ 27/May/15 ]

Hi tryshi,

Thank you so much for providing access. It made this go much faster.

And good news! I've figured out the problem. In your connection string, your second host has a space before it just after the comma. So, mongodb://badhost, goodhost doesn't work because it's trying to resolve the "goodhost" server with " goodhost". Simply removing the comma fixes the problem. Also, just to note, using mongodb://goodhost, badhost works as well because "goodhost" is resolved correctly.

We can see this from the socket logs:

System.Net.Sockets Verbose: 0 : [1324] Socket#12919499::Socket(AddressFamily#2)
System.Net.Sockets Verbose: 0 : [1324] Exiting Socket#12919499::Socket() 
System.Net.Sockets Verbose: 0 : [1324] Socket#12919499::BeginConnect( vmsec0521e.westus.cloudapp.azure.com)
System.Net.Sockets Verbose: 0 : [1324] DNS::UnsafeBeginGetHostAddresses( vmsec0521e.westus.cloudapp.azure.com)
System.Net.Sockets Verbose: 0 : [1324] Exiting DNS::UnsafeBeginGetHostAddresses() 	-> ResolveAsyncResult#49891972
System.Net.Sockets Verbose: 0 : [5260] DNS::GetHostByName( vmsec0521e.westus.cloudapp.azure.com)
System.Net.Sockets Verbose: 0 : [1324] Exiting Socket#12919499::BeginConnect() 	-> MultipleAddressConnectAsyncResult#23511423
System.Net.Sockets Verbose: 0 : [5260] DNS::EndGetHostAddresses(ResolveAsyncResult#49891972)
System.Net.Sockets Verbose: 0 : [4376] Socket#12919499::EndConnect(MultipleAddressConnectAsyncResult#23511423)
System.Net.Sockets Error: 0 : [4376] Exception in Socket#12919499::EndConnect - No such host is known.
   at System.Net.Dns.HostResolutionEndHelper(IAsyncResult asyncResult)
   at System.Net.Dns.EndGetHostAddresses(IAsyncResult asyncResult)
   at System.Net.Sockets.Socket.DnsCallback(IAsyncResult result)

As well as our own logs:

2015-05-27 14:35:17.2377021Z Info  1   1: added server  vmsec0521e.westus.cloudapp.azure.com:37000 in 0.7116ms.
2015-05-27 14:35:17.2396908Z Info  1   1: { ClusterId : "1", Type : "ReplicaSet", State : "Disconnected", Servers : [{ ServerId: "{ ClusterId : 1, EndPoint : "Unspecified/ vmsec0521e.westus.cloudapp.azure.com:37000" }", EndPoint: "Unspecified/ vmsec0521e.westus.cloudapp.azure.com:37000", State: "Disconnected", Type: "Unknown" }, { ServerId: "{ ClusterId : 1, EndPoint : "Unspecified/vmpri0521e-wrong-dns.westus.cloudapp.azure.com:37000" }", EndPoint: "Unspecified/vmpri0521e-wrong-dns.westus.cloudapp.azure.com:37000", State: "Disconnected", Type: "Unknown" }] }
2015-05-27 14:35:17.2426911Z Error 8   1: vmsec0521e.westus.cloudapp.azure.com:37000:2: unable to open. Exception: MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server. ---> System.Net.Sockets.SocketException: No such host is known
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)

I am currently chatting with our other driver engineers and spec authors to determine whether or not we should be stripping whitespace. Apparently some do and some don't. If we decide to do that, I'll open another ticket specifically for it.

For now, could you confirm that this fixes your problems?
Craig

Comment by tryshi [ 27/May/15 ]

The repro needs below workaround with hosts file in C:\Windows\System32\drivers\etc

#Db IP host mapping
191.236.92.13 vmpri0521e
191.236.81.200 vmsec0521e
191.239.49.7 vmarb0521e

Comment by tryshi [ 27/May/15 ]

The logs are in the attachment.

here is the code with real username/password populated, so you can easily repro it by yourself.

            // Good-TwoSeeds
             var cnnString = "mongodb://<userName:<userPassword>@vmpri0521e.westus.cloudapp.azure.com:27000, vmsec0521e.westus.cloudapp.azure.com:27000/testdb?replicaSet=rs0521e";
 
            // Good-SingleSeed
            // var cnnString = "mongodb://<userName>:<userPassword>@vmsec0521e.westus.cloudapp.azure.com:27000/testdb?replicaSet=rs0521e";
 
            // Bad-TwoSeeds-With-wrong-dns
            //var cnnString = "mongodb://<userName>:<password>@vmpri0521e-wrong-dns.westus.cloudapp.azure.com:27000, vmsec0521e.westus.cloudapp.azure.com:37000/testdb?replicaSet=rs0521e";
 
            var settings = MongoClientSettings.FromUrl(new MongoUrl(cnnString));
            settings.ClusterConfigurator = cb =>
            {
                var textWriter = TextWriter.Synchronized(new StreamWriter("mylogfile.txt"));
                cb.AddListener(new LogListener(textWriter));
            };
 
            var client = new MongoClient(settings);
            var db = client.GetDatabase("testdb");
            var col = db.GetCollection<BsonDocument>("users");
            await col.InsertOneAsync(new BsonDocument("name", "jack"));
            var users = await col.Find(e => true).ToListAsync();
            foreach (var document in users)
            {
                Console.WriteLine(document["name"]);
            }

Comment by tryshi [ 27/May/15 ]

Here are the logs.

Comment by tryshi [ 27/May/15 ]

Thanks Craig for your time.
I will provide the information later tonight.

tryshi.

Comment by Craig Wilson [ 27/May/15 ]

Thanks for confirming you can connect from the shell. The driver doesn't look at priority, but that's certainly a good guess.

1. Are you also able to connect from the driver when only supplying that one host? For example, using this connection string: mongodb://vmsec0521e.westus.cloudapp.azure.com:37000.

2. If not, it'd be great if you could provide a .NET socket trace log as specified here: https://msdn.microsoft.com/en-us/library/ty48b824%28v=vs.110%29.aspx. In particular, I'm looking for the System.Net.Sockets output.

3. Also, if you're up for it, I'd love to see the log when you configure tracing from the driver itself:

var settings = MongoClientSettings.FromUrl(new MongoUrl(connectionString));
settings.ClusterConfigurator = cb =>
{
    var textWriter = TextWriter.Synchronized(new StreamWriter("mylogfile.txt"));
    cb.AddListener(new LogListener(textWriter));
};
 
var client = new MongoClient(settings);
// etc...

Thanks for hanging with me.
Craig

Comment by tryshi [ 27/May/15 ]

My hunch is the Priority may play a role here. .

Comment by tryshi [ 27/May/15 ]

I can always connect to the vmsec0512e. below is the result after failover.

mongo.exe vmsec0521e.westus.cloudapp.azure.com:37000/testoc -u <userName> -p <passWord>
MongoDB shell version: 3.0.2
connecting to: vmsec0521e.westus.cloudapp.azure.com:37000/testoc
rs0521e:PRIMARY>

Comment by tryshi [ 27/May/15 ]

for your question:
>>are you able to use the mongo shell and successfully connect?
>>mongo --host vmsec0521e.westus.cloudapp.azure.com --port 37000

Yes, I can.

Comment by tryshi [ 27/May/15 ]

repro by killing mongod process in Primary vmpri0521e

Code, same as above, execept connection string is as the following.

var cnnString = "mongodb://<userName>:<passWord>@vmpri0521e.westus.cloudapp.azure.com:37000, vmsec0521e.westus.cloudapp.azure.com:37000/testoc?replicaSet=rs0521e";

ssh into the Primary machine vmpri0512e, kill mongod process.
Waited for mongodb to fail over to secondary(verified by NodeJS code and shell).
run the code.

Exception

Unhandled Exception: System.AggregateException: One or more errors occurred. ---> System.TimeoutException: A timeout occured after 30000ms selecting a server using CompositeServerSelector{ Selectors = ReadPreferenceServerSelector{ ReadPreference =

{ Mode = Primary, TagSets = System.Collections.Generic.List`1[MongoDB.Driver.TagSet] }

}, LatencyLimitingServerSelector

{ AllowedLatencyRange = 00:00:00.0150000 }

}. Client view of cluster state is { ClusterId : "1", Type : "ReplicaSet", State : "Disconnected", Servers : [{ ServerId: "

{ ClusterId : 1, EndPoint : "Unspecified/ vmsec0521e.westus.cloudapp.azure.com:37000" }

", EndPoint: "Unspecified/ vmsec0521e.westus.cloudapp.azure.com:37000", State: "Disconnected", Type: "Unknown", HeartbeatException: "MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server. ---> System.Net.Sockets.SocketException: No such host is known
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Connections.TcpStreamFactory.<ConnectAsync>d__7.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Connections.TcpStreamFactory.<CreateStreamAsync>d__0.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at MongoDB.Driver.Core.Connections.BinaryConnection.<OpenAsyncHelper>d__1.MoveNext()
— End of inner exception stack trace —
at MongoDB.Driver.Core.Connections.BinaryConnection.<OpenAsyncHelper>d__1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Servers.ClusterableServer.<HeartbeatAsync>d__d.MoveNext()" }, { ServerId: "

{ ClusterId : 1, EndPoint : "Unspecified/vmpri0521e.westus.cloudapp.azure.com:37000" }

", EndPoint: "Unspecified/vmpri0521e.westus.cloudapp.azure.com:37000", State: "Disconnected", Type: "Unknown", HeartbeatException: "MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server. ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 191.236.92.13:37000
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Connections.TcpStreamFactory.<ConnectAsync>d__7.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Connections.TcpStreamFactory.<CreateStreamAsync>d__0.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at MongoDB.Driver.Core.Connections.BinaryConnection.<OpenAsyncHelper>d__1.MoveNext()
— End of inner exception stack trace —
at MongoDB.Driver.Core.Connections.BinaryConnection.<OpenAsyncHelper>d__1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Servers.ClusterableServer.<HeartbeatAsync>d__d.MoveNext()" }] }.
at MongoDB.Driver.Core.Clusters.Cluster.ThrowTimeoutException(IServerSelector selector, ClusterDescription description)
at MongoDB.Driver.Core.Clusters.Cluster.<WaitForDescriptionChangedAsync>d__18.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Clusters.Cluster.<SelectServerAsync>d__6.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Bindings.ReadPreferenceBinding.<GetReadChannelSourceAsync>d__0.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at MongoDB.Driver.Core.Operations.FindOperation`1.<ExecuteAsync>d__2.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.OperationExecutor.<ExecuteReadOperationAsync>d__0`1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.MongoCollectionImpl`1.<ExecuteReadOperation>d__35`1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.IAsyncCursorSourceExtensions.<ToListAsync>d__14`1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
at mdb.Program.<Run>d__4.MoveNext() in c:\vso\ops\svcclient\mdb\cs\Program.cs:line 62
— End of inner exception stack trace —
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Task.Wait()
at mdb.Program.Main(String[] args) in c:\vso\ops\svcclient\mdb\cs\Program.cs:line 24

Comment by Craig Wilson [ 27/May/15 ]

Thanks. In this particular example, it's indicating that we cannot talk to the good host either. Perhaps this is what you are reporting, but this exception ultimately comes from the .NET framework:

...
EndPoint: "Unspecified/ vmsec0521e.westus.cloudapp.azure.com:37000", State: "Disconnected", Type: "Unknown", HeartbeatException: "MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server. ---> System.Net.Sockets.SocketException: No such host is known
...

Because we can't connect to it, we aren't able to find the other ones either. Can you confirm that if you only connect to this one machine (mongodb://vmsec0521e.westus.cloudapp.azure.com:37000), you are able to talk to it? Also, are you able to use the mongo shell and successfully connect?
mongo --host vmsec0521e.westus.cloudapp.azure.com --port 37000

Craig

Comment by tryshi [ 27/May/15 ]

I thought the scenarios is straight forward enough, so I omitted some unimportant details, but here is the full bug report.

Build a replication set as the following

machine vmpri0512e: primary (priority 5)
machine vmsec0512e: secondary (priority 1)
machine vmarb0512e: arbeiter

Repro with non existent dns name.

Put a invalid(not exist ) dns name in the seed list. Note: I've already changed the etc/hosts file to map the private host name with IP.

Code:

var myConventions = new ConventionPack();
myConventions.Add(new IgnoreExtraElementsConvention(true));
myConventions.Add(new IgnoreIfNullConvention(true));
ConventionRegistry.Register("betterConvention", myConventions, (type) => true);

var cnnString = "mongodb://<userName>:<passWord>@non-existent-dns.westus.cloudapp.azure.com:37000, vmsec0521e.westus.cloudapp.azure.com:37000/testoc?replicaSet=rs0521e";
var client = new MongoClient(cnnString);
var db = client.GetDatabase("testoc");
var col = db.GetCollection<BsonDocument>("appUsers");
var users = await col.Find(e => true).ToListAsync();

Exception:

Unhandled Exception: System.AggregateException: One or more errors occurred. ---> System.TimeoutException: A timeout occured after 30000ms selecting a server using CompositeServerSelector{ Selectors = ReadPreferenceServerSelector{ ReadPreference =

{ Mode = Primary, TagSets = System.Collections.Generic.List`1[MongoDB.Driver.TagSet] }

}, LatencyLimitingServerSelector

{ AllowedLatencyRange = 00:00:00.0150000 }

}. Client view of cluster state is { ClusterId : "1", Type : "ReplicaSet", State : "Disconnected", Servers : [{ ServerId: "

{ ClusterId : 1, EndPoint : "Unspecified/ vmsec0521e.westus.cloudapp.azure.com:37000" }

", EndPoint: "Unspecified/ vmsec0521e.westus.cloudapp.azure.com:37000", State: "Disconnected", Type: "Unknown", HeartbeatException: "MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server. ---> System.Net.Sockets.SocketException: No such host is known
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Connections.TcpStreamFactory.<ConnectAsync>d__7.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Connections.TcpStreamFactory.<CreateStreamAsync>d__0.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at MongoDB.Driver.Core.Connections.BinaryConnection.<OpenAsyncHelper>d__1.MoveNext()
— End of inner exception stack trace —
at MongoDB.Driver.Core.Connections.BinaryConnection.<OpenAsyncHelper>d__1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Servers.ClusterableServer.<HeartbeatAsync>d__d.MoveNext()" }, { ServerId: "

{ ClusterId : 1, EndPoint : "Unspecified/non-existent-dns.westus.cloudapp.azure.com:37000" }

", EndPoint: "Unspecified/non-existent-dns.westus.cloudapp.azure.com:37000", State: "Disconnected", Type: "Unknown", HeartbeatException: "MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server. ---> System.Net.Sockets.SocketException: No such host is known
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Connections.TcpStreamFactory.<ConnectAsync>d__7.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Connections.TcpStreamFactory.<CreateStreamAsync>d__0.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at MongoDB.Driver.Core.Connections.BinaryConnection.<OpenAsyncHelper>d__1.MoveNext()
— End of inner exception stack trace —
at MongoDB.Driver.Core.Connections.BinaryConnection.<OpenAsyncHelper>d__1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Servers.ClusterableServer.<HeartbeatAsync>d__d.MoveNext()" }] }.
at MongoDB.Driver.Core.Clusters.Cluster.ThrowTimeoutException(IServerSelector selector, ClusterDescription description)
at MongoDB.Driver.Core.Clusters.Cluster.<WaitForDescriptionChangedAsync>d__18.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Clusters.Cluster.<SelectServerAsync>d__6.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Bindings.ReadPreferenceBinding.<GetReadChannelSourceAsync>d__0.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at MongoDB.Driver.Core.Operations.FindOperation`1.<ExecuteAsync>d__2.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.OperationExecutor.<ExecuteReadOperationAsync>d__0`1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.MongoCollectionImpl`1.<ExecuteReadOperation>d__35`1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.IAsyncCursorSourceExtensions.<ToListAsync>d__14`1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
at mdb.Program.<Run>d__2.MoveNext() in c:\vso\ops\svcclient\mdb\cs\Program.cs:line 39
— End of inner exception stack trace —
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Task.Wait()
at mdb.Program.Main(String[] args) in c:\vso\ops\svcclient\mdb\cs\Program.cs:line 24

Comment by Craig Wilson [ 27/May/15 ]

Hi tryshi,

It is very difficult to understand what is going on from these repros. I am looking for something more like: 1) initiate a 3 member replica set. 2) use the connection string "mongodb://...". 3) XXX

Also, this does not look like the entire stack trace. Is there more that wasn't pasted in? For instance, I only see a single host in here when there is obviously 2. It would appear as though we simply can't talk to any member, which relates back to your other ticket (CSHARP-1294). In addition, if you kill a primary when using the ReadPreference of Primary (the default), then until a new primary is elected, you won't be able to read.

I'm sorry this is so difficult to ascertain what is going on. Like I said, we test these scenarios extensively, so diagnosing this is going to take some time.

Craig

Comment by tryshi [ 27/May/15 ]

Repro with non existent dns name:

connection string: "mongodb://<userName>:<passWord>@non-existent-dns.westus.cloudapp.azure.com:27000, vmsec0521e.westus.cloudapp.azure.com:27000/testoc?replicaSet=rs0521e";

System.TimeoutException: A timeout occured after 30000ms selecting a server using CompositeServerSelector{ Selectors = ReadPreferenceServerSelector{ ReadPreference =

{ Mode = Primary, TagSets = System.Collections.Generic.List`1[MongoDB.Driver.TagSet] }

}, LatencyLimitingServerSelector

{ AllowedLatencyRange = 00:00:00.0150000 }

}. Client view of cluster state is { ClusterId : "1", Type : "ReplicaSet", State : "Disconnected", Servers : [{ ServerId: "

{ ClusterId : 1, EndPoint : "Unspecified/ vmsec0521e.westus.cloudapp.azure.com:27000" }

", EndPoint: "Unspecified/ vmsec0521e.westus.cloudapp.azure.com:27000", State: "Disconnected", Type: "Unknown", HeartbeatException: "MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server. ---> System.Net.Sockets.SocketException: No such host is known
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)

repro by killing mongod process in Primary vmpri0521e

connection string: "mongodb://<userName>:<passWord>@vmpri0521e.westus.cloudapp.azure.com:27000, vmsec0521e.westus.cloudapp.azure.com:27000/testoc?replicaSet=rs0521e";

at MongoDB.Driver.Core.Servers.ClusterableServer.<HeartbeatAsync>d__d.MoveNext()" }, { ServerId: "

{ ClusterId : 1, EndPoint : "Unspecified/vmpri0521e.westus.cloudapp.azure.com:27000" }

", EndPoint: "Unspecified/vmpri0521e.westus.cloudapp.azure.com:27000", State: "Disconnected", Type: "Unknown", HeartbeatException: "MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server. ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 191.236.92.13:27000

Comment by Craig Wilson [ 25/May/15 ]

Hi tryshi,

We certainly don't require all seeds to be available. If that is happening, it would be a bug. This is a scenario we test, so I'm keen to understand what is going on. Could you provide your connection string/configuration code and/or provide a simple set of code we can reproduce this with.

Craig

Generated at Wed Feb 07 21:39:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.