[CSHARP-1663]  MongoDB.Driver.Core.Clusters.Cluster.ThrowTimeoutException Created: 12/May/16  Updated: 05/Apr/19  Resolved: 22/Jan/18

Status: Closed
Project: C# Driver
Component/s: Connectivity
Affects Version/s: 2.2.3
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: TD [X] Assignee: Robert Stam
Resolution: Cannot Reproduce Votes: 0
Labels: question
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows / C# / .NET 4.0.30319


Issue Links:
Related
related to CSHARP-2147 Add logging of heartbeat information ... Closed

 Description   

In this scenario, there are two computers with a 1Gbps private link.
MongoDB runs on one and the application having problems on the other one.

After a few days of running with roughly 10-15 transactions / seconds, I am getting Timeout errors.

The database replies properly on its host.
The link between the two machine is working properly.

There is no replica set, etc, just a single database.
CPU usage is low, there is plenty of memory; that on both computers
I can't figure out what causes this problem; here is the log:

Exception Info: System.TimeoutException
at MongoDB.Driver.Core.Clusters.Cluster.ThrowTimeoutException(MongoDB.Driver.Core.Clusters.ServerSelectors.IServerSelector, MongoDB.Driver.Core.Clusters.ClusterDescription)
at MongoDB.Driver.Core.Clusters.Cluster+WaitForDescriptionChangedHelper.HandleCompletedTask(System.Threading.Tasks.Task)
at MongoDB.Driver.Core.Clusters.Cluster+<WaitForDescriptionChangedAsync>d__44.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
at MongoDB.Driver.Core.Clusters.Cluster+<SelectServerAsync>d__37.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
at MongoDB.Driver.Core.Bindings.WritableServerBinding+<GetWriteChannelSourceAsync>d__8.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
at MongoDB.Driver.Core.Operations.CreateIndexesOperation+<ExecuteAsync>d__15.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
at MongoDB.Driver.OperationExecutor+<ExecuteWriteOperationAsync>d__3`1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
at MongoDB.Driver.MongoCollectionImpl`1+<ExecuteWriteOperationAsync>d__61`1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
at MongoDB.Driver.MongoCollectionImpl`1+MongoIndexManager+<CreateManyAsync>d__9[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
at MongoDB.Driver.MongoIndexManagerBase`1+<CreateOneAsync>d__7[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].MoveNext()



 Comments   
Comment by Robert Stam [ 22/Jan/18 ]

Linking to CSHARP-2147 as additional logging would have helped troubleshoot the potential issue.

Comment by TD [X] [ 13/May/16 ]

Hi Craig,

I don't have the logs with the exception anymore; unfortunately this one happens after a few days, so I will have to wait for the next instance.

On the same system, I also get a queue full exception regularly;I wonder if it is possible that both are related as multiple services talk to the DB; possibly one times out if the queue is full causing the error from above.

This is what I get regularly:
Inner Exception System.AggregateException: One or more errors occurred. ---> MongoDB.Driver.MongoWaitQueueFullException: The wait queue for acquiring a connection to server brain:27017 is full.
at MongoDB.Driver.Core.ConnectionPools.ExclusiveConnectionPool.AcquireConnectionHelper.CheckingOutConnection()
at MongoDB.Driver.Core.ConnectionPools.ExclusiveConnectionPool.<AcquireConnectionAsync>d__35.MoveNext()

I have this in the connection string: ?maxPoolSize=10000 so I am assuming the queue is large enough.

As the API has now been async for a while, the code just throws requests whenever it needs them.

Comment by Craig Wilson [ 12/May/16 ]

Hi TD,

Sorry you are having some trouble. Could you provide the exception message, not just the exception stacktrace? The message contains some valuable information about the state of the current cluster. Also, if you could provide the client.Cluster.ClusterDescription.ToString() value at the time you are getting the exception, that could be helpful as well.

Thanks,
Craig

Generated at Wed Feb 07 21:40:19 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.