[CSHARP-204] Concurrency Issue on failure recovery Created: 15/Apr/11 Updated: 02/Apr/15 Resolved: 06/Jun/11 |
|
| Status: | Closed |
| Project: | C# Driver |
| Component/s: | None |
| Affects Version/s: | 1.0 |
| Fix Version/s: | 1.1 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Miguel Pilar | Assignee: | Robert Stam |
| Resolution: | Done | Votes: | 1 |
| Labels: | concurrency | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Windows 2003, IIS, connected to a single mongos instance |
||
| Attachments: |
|
| Backwards Compatibility: | Fully Compatible |
| Description |
|
Note: using git revision 1a730b for line numbers. While doing high-concurrency load tests (~2000 connections) against our web service which calls mongo I ran into a concurrency issue that narrowed down to a problem between MongoServer.Disconnect() and MongoServer.AcquireConnection(MongoDatabase database,bool slaveOk). The issue occurs when an exception occurs within a MongoConnection between the GetConectionPoolEndPoint call (MongoServer.cs:798) and the connectionPool.AcquireConnection(database) (MongoServer.cs:807) call in AcquireConnection(database, endPoint) (MongoServer.cs:799). Since MongoConnection.HandleException() is forcing a server disconnect, when connectionPool.AcquireConnection(database) gets called, the result is eventually a InvalidOperationException("Attempt to get a connection from a closed connection pool") exception (MongoConnectionPool.cs:103). As this is a concurrency issue that I've only seen in a very specific test, I have been unable to write a reliable test for it (we have to saturate the data link with several Gbps of data to make it fail semi-consistently). Attached is the patch that I've implemented to fix this issue, since the lock() calls are reentrant we simply wrap both GetConectionPoolEndPoint and AcquireConnection with a lock(serverLock). So far this code has performed well, but it needs to be thoroughly reviewed to make sure my simple fix will not cause deadlocks on some scenario I have not taken into account. This bug possibly affects previous versions. |
| Comments |
| Comment by Robert Stam [ 06/Jun/11 ] |
|
Reviewed the patch and agreed with it. Thanks Miguel! |
| Comment by Grégoire Seux [ 23/May/11 ] |
|
This issue also occured on Linux with mongo 1.8.1 and the last stable c# driver. |