[CSHARP-1302] MongoClient Connection issues. Created: 08/Jun/15  Updated: 05/Apr/19  Resolved: 08/Jun/15

Status: Closed
Project: C# Driver
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Paul Reed Assignee: Unassigned
Resolution: Done Votes: 0
Labels: question
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

I have an issue with connections failing in a certain scenario:

Architecture:

website#1 with primary#1
website#2 with secondary#2
website#3 with secondary#3

Settings: new MongoClientSettings

{ ReplicaSetName = "rs", Servers = < 3 server names >, ConnectionMode = ConnectionMode.ReplicaSet, ConnectTimeout = new TimeSpan(0,0,60), ReadPreference = (slaveOk) ? ReadPreference.Nearest : ReadPreference.Primary, WriteConcern = WriteConcern.Acknowledged, }


I run this command:
var client = MongoClient(Settings);
await client.GetDatabase("admin").RunCommandAsync("

{replSetGetStatus:1}

")

and generally receive a correct response with myState either being 1 or 2.

However, sometimes whilst these sites are running I will take offline secondary#2 - and run up a local non-rs instance on port 37017. When I do that, website#2 begins to respond with:

A timeout occured after 30000ms selecting a server using CompositeServerSelector{ Selectors = WritableServerSelector, LatencyLimitingServerSelector

{ AllowedLatencyRange = 00:00:00.0150000 }

}. Client view of cluster state is { ClusterId : "1", Type : "ReplicaSet", State : "Disconnected", Servers : [{ ServerId: "

{ ClusterId : 1, EndPoint : "Unspecified/127.0.0.1:27017" }

", ....

Now - I would have expected the website to fall over to retrieve secondary#3 - but its not doing that.

Am I doing something wrong?
Should perhaps my MongoClient be more globalized within the AppDomain (maybe - but is it related).

Hope you can help.



 Comments   
Comment by Craig Wilson [ 08/Jun/15 ]

In most cases, it won't matter because if the settings are the same, we'll use the same cluster underneath. However, this isn't always true because there are certain types of settings we cannot compare.

To be safe, it is always best practice to use a single MongoClient for your application. Stick it in your IoC container or something.

Since we've figured this out, I'm going to go ahead and close this ticket.
Craig

Comment by Paul Reed [ 08/Jun/15 ]

Oh Golly,

Thank you:

#if !DEBUG
useLocal = true;
#endif

...

if (useLocal)

{ servers.Add(new MongoServerAddress("127.0.0.1", port)); }

Cannot believe that this has been added in at some stage - it used to be #if DEBUG.. - and of course the port is coming in at the correct value. So amazingly every thing works out normally.

#redface

===

So, just one last thing. When using MongoClient/MongoServer within a WebSite ( utilising ApplicationPooling ) - should we be using a single MongoClient/Server per app pool instance ?
Or does it actually not matter if we generate many new MongoClient(settings) calls.

Comment by Craig Wilson [ 08/Jun/15 ]

Right. Are you giving it 127.0.0.1 anywhere in your code? For some reason, we aren't connecting to a replica set here, just the local mongod. I've attempted to replicate this and am not having any luck. Somehow, the 3 servers you are indicating in your MongoClientSettings are not getting used. Is there anyway you could provide a reproduction of this with actual code so that I can debug through it?

Craig

Comment by Paul Reed [ 08/Jun/15 ]

If it is trying to connect to the Primary, then I really am not sure why it is using: 127.0.0.1:27017 as this is the local address to the secondary. Non of the server addresses are in IP format.
The server addresses are presented within the settings in the order Primary#1,Secondary#2, Secondary#3

Comment by Paul Reed [ 08/Jun/15 ]

re: 1)
we have websites and mongod running across 3 servers. The websites are not heavy load - all the load is within mongod.

re: 2)

A timeout occured after 30000ms selecting a server using CompositeServerSelector{ Selectors = WritableServerSelector, LatencyLimitingServerSelector

{ AllowedLatencyRange = 00:00:00.0150000 }

}. Client view of cluster state is { ClusterId : "1", Type : "ReplicaSet", State : "Disconnected", Servers : [{ ServerId: "

{ ClusterId : 1, EndPoint : "Unspecified/127.0.0.1:27017" }

", EndPoint: "Unspecified/127.0.0.1:27017", State: "Disconnected", Type: "Unknown", HeartbeatException: "MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server. ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 127.0.0.1:27017
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Connections.TcpStreamFactory.d__7.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Connections.TcpStreamFactory.d__0.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at MongoDB.Driver.Core.Connections.BinaryConnection.d__1.MoveNext()
— End of inner exception stack trace —
at MongoDB.Driver.Core.Connections.BinaryConnection.d__1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Servers.ClusterableServer.d__d.MoveNext()" }] }.
at MongoDB.Driver.Core.Clusters.Cluster.ThrowTimeoutException(IServerSelector selector, ClusterDescription description)
at MongoDB.Driver.Core.Clusters.Cluster.d__18.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Clusters.Cluster.d__6.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Bindings.WritableServerBinding.d__0.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.Core.Operations.WriteCommandOperation`1.d__0.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.OperationExecutor.d__3`1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MongoDB.Driver.MongoDatabaseImpl.d__7`1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at #########.MongoDataConnection.<>c_DisplayClass20.<b1b>d_22.MoveNext()
===

re: 3)

The primary is up at all times,
website#2 is still up, but has no secondary running on it - just a local instance running non-rs.
secondary#2 was cleanly brought down off of 27017 - and then a local instance ran up on 37017.
website#1 and website#3 also run this command and respond with no issue's.
All the function is doing is to try and ascertain if I have a functional replicaset - I have had problems in the past with the sites being ready before mongo has had time to come up into a ready state. When this happens I correctly get the timeout which holds up my site processing correctly. I

Comment by Craig Wilson [ 08/Jun/15 ]

Hi Paul,

Sorry you are having issues. Couple of questions:

1. I don't completely understand what you've said about your architecture. Are you running your mongod instances on the same servers as your website servers?
2. In the exception message you've provided, you didn't include the entire message. Could you provide the entire message. It let's us know exactly what the state of the cluster is when the timeout failed.
3. In the exception message you've provided, it is attempting to find a Primary. The fact that it didn't means that no primary was available. (WritableServerSelector = find a primary). When you run a command, it will always use a read preference of primary regardless of your settings. There is an optional parameter on RunCommandAsync which allows you to provide a read preference: http://api.mongodb.org/csharp/2.0/html/M_MongoDB_Driver_IMongoDatabase_RunCommandAsync__1.htm.

Craig

Comment by Paul Reed [ 08/Jun/15 ]

FYI

  • useSlave == true
    so the ReadPreference is Nearest
Generated at Wed Feb 07 21:39:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.