[CSHARP-1303] Socket Connection Error in 2.0 Driver Created: 09/Jun/15 Updated: 15/Nov/21 Resolved: 04/Apr/16 |
|
| Status: | Closed |
| Project: | C# Driver |
| Component/s: | Connectivity, Error Handling |
| Affects Version/s: | 2.0 |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Bret Ferrier | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 2 |
| Labels: | question | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Azure website connecting to a Linux replica set on Azure |
||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Description |
|
Since going through and updating all of my code to use the mongodb 2.0 driver when I look in my logs I see hundreds of error messages just like the one below where the driver is getting a socket error when connecting to mongo. When I look at the logs on the mongo server I do not see any error messages. The site is still up and this error doesn't happen on every request, nor does it happen on one operation that should take longer. It is random where it occurs within the application. Here is the error: MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server. ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 138.91.138.169:27017 |
| Comments |
| Comment by John Murphy [ 16/Sep/19 ] | |||||||||||||||
|
If you are unable to enable TCP keep alives at the Operating System or Socket level, you can set the MaxConnectionIdleTime to instruct the MongoDB C#/.NET driver to not use pooled connections that may have been severed by the Azure Network Load Balancer. This approach however will not proactively detect connections that have been reset unexpectedly, and it will not prevent connections with long running operations from being severed by Azure. Therefore we suggest that you utilise the TCP keep alive method when using the MongoDB C#/.NET driver within Azure environments to provide the highest level of connection resilience. | |||||||||||||||
| Comment by Sebastian Inones [ 12/Sep/19 ] | |||||||||||||||
|
I'm facing the same issue using App Service hosted on Azure connected to a MongoDB Server outside Azure on a different place.
if as he mentioned "environments such as Azure sever idle TCP connections after 4 minutes." ? | |||||||||||||||
| Comment by John Murphy [ 20/Jun/19 ] | |||||||||||||||
|
Note that our production notes state that environments such as Azure sever idle TCP connections after 4 minutes. Therefore in these environments the keep alive interval needs to be configured to be 120 seconds. This comment on | |||||||||||||||
| Comment by Vadim Rybak [ 30/Aug/17 ] | |||||||||||||||
|
Even this workaround doesn't seem to be working for us.
We also confirmed that our keep alive on the box hosting our code is set correctly We are still getting the same issues that Jeremy was. Are there any other workarounds that we can do? | |||||||||||||||
| Comment by Jeffrey Yemin [ 29/Aug/17 ] | |||||||||||||||
|
jstafford@digitalairstrike.com thanks for posting your workaround, and also please watch this related ticket to enable TCP Keep-Alive by default: | |||||||||||||||
| Comment by Jeremy Stafford [ 29/Aug/17 ] | |||||||||||||||
|
For what it's worth, it appears that the driver doesn't enable socket keep alive by default (strange considering this is a database connection...). The following solved it for us:
I'm surprised the MongoDB team isn't taking this more seriously. This stuff should be perfect for common use out of the box. To be fair though, apparently MSSQL had the same issue in Azure a while back and solved it in a similar fashion in the ADO driver. | |||||||||||||||
| Comment by Jaume Bresco Arroyo [ 29/Aug/17 ] | |||||||||||||||
|
We are experiencing the same issue, why is closed? Thanks | |||||||||||||||
| Comment by Kamil Mackow [X] [ 28/Jul/17 ] | |||||||||||||||
|
We are still experiencing this issue in Azure and have tried all suggestions thus far to no avail. Why is this closed? | |||||||||||||||
| Comment by Wayne He [ 06/Jul/17 ] | |||||||||||||||
|
I wonder why this bug is marked as Closed. | |||||||||||||||
| Comment by Jeremy Stafford [ 10/Feb/17 ] | |||||||||||||||
|
So by my reading, Azure's default timeout is 4 minutes. I set maxIdleTime to 60000 and 10000 and still getting the same problem. | |||||||||||||||
| Comment by Craig Wilson [ 03/Feb/16 ] | |||||||||||||||
|
Sinan, Azure has a low setting for idle connections and kills them when they've hit that limit. You need to specify either maxIdleTime on the connection string or MaxConnectionIdleTime in MongoClientSettings to something around 3 minutes. That way, the driver won't use a connection that has been idle for too long and may have been killed off by azure already. Craig | |||||||||||||||
| Comment by Sinan Bir [ 03/Feb/16 ] | |||||||||||||||
|
I have Linux replica set on Azure and my front-end application deployed as a cloud service in Azure. When I send request after a while ( about 10 minutes later after first request ) I always get socket exception error. [SocketException (0x274c): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond] [IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.] [MongoConnectionException: An exception occurred while receiving a message from the server.] | |||||||||||||||
| Comment by Roberto Pérez [ 02/Feb/16 ] | |||||||||||||||
|
Same issue here. Error Message: Stack Trace: Thank you! | |||||||||||||||
| Comment by Bret Ferrier [ 24/Aug/15 ] | |||||||||||||||
|
Ahmet, I ended up putting the MaxConnectionIdleTime around 45 seconds and then had to create a set of Extension Methods off of the IFindFluent Interface that included retry logic. In my testing trying the same query right after the exception was received just about always worked. Really seems that the retry logic or error handling should be baked in but it is not and doesn’t seem that anyone is looking at it. From: ahmet taha sakar (JIRA) ahmet taha sakar commented on I am also having the same issue. I'm on azure. I have default MaxConnectionIdleTime set to 1 minutes from the mongo c# driver. ---------------------- | |||||||||||||||
| Comment by Ahmet Taha sakar [ 21/Aug/15 ] | |||||||||||||||
|
I am also having the same issue. I'm on azure. I have default MaxConnectionIdleTime set to 1 minutes from the mongo c# driver. | |||||||||||||||
| Comment by Bret Ferrier [ 09/Jun/15 ] | |||||||||||||||
|
Below is the connection string <add name="mongoEvents" connectionString="mongodb://remote:7CrazyIdea@xxmongolinux1.cloudapp.net:27017,xxmongolinux2.cloudapp.net:27017/Events?connectTimeoutMS=30000&socketTimeoutMS=30000&waitQueueTimeoutMS=30000&maxIdleTimeMS=45000" /> As you can see I am already setting the maxIdleTimeMS, On another site that we have I found that setting the maxIdleTime to 60000 didn't actually help but 45000 did solve the idle connection issue on Azure but this seems different. -Bret | |||||||||||||||
| Comment by Craig Wilson [ 09/Jun/15 ] | |||||||||||||||
|
What does your connection string look like? Or your in code setup? Azure has something that tends to kill off idle connections. Please try setting the maxIdleTimeMS connection string option to 60000 (1 minute). You can also set this in code using the MaxConnectionIdleTime property on MongoClientSettings. Hopefully, this will clear it up. Craig | |||||||||||||||
| Comment by Bret Ferrier [ 09/Jun/15 ] | |||||||||||||||
|
Pretty much all of the exceptions I am seeing are the Socket exceptions. Also this is running as an Azure website. Is there anything I can do with the Mongo driver to handle these exceptions? | |||||||||||||||
| Comment by Craig Wilson [ 09/Jun/15 ] | |||||||||||||||
|
Hi Bret, Sorry you are having issues. This exception is coming straight out of the .NET framework's Socket class. I assume one your mongod servers is sitting at 138.91.138.169:27017. Generally when this exception bubbles to user code, it means that the network is a little flaky. Otherwise, you would also probably be seeing a number of TimeoutExceptions in the logs as well. Is this the only exception you are seeing or are you also seeing TimeoutExceptions? Also, are you using Azure or AWS? Craig |