[CSHARP-268] NullReferenceException on lost primary Created: 13/Jul/11 Updated: 02/Apr/15 Resolved: 01/Aug/11 |
|
| Status: | Closed |
| Project: | C# Driver |
| Component/s: | None |
| Affects Version/s: | 1.2 |
| Fix Version/s: | 1.2 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Aristarkh Zagorodnikov | Assignee: | Robert Stam |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
git version d7ce7f2ee560183d8031 |
||
| Issue Links: |
|
||||||||||||||||
| Description |
|
3-server replica set (primary, slave, arbiter). Primary steps down, the next query (actually any queries) fail with NullReferenceException with the following stack trace: It appears that connectionPool is null, probably because MongoServerInstance.Disconnect due to the connection error. Might be related to |
| Comments |
| Comment by Aristarkh Zagorodnikov [ 02/Aug/11 ] |
|
My default RS setup: primary, secondary, arbiter; slaveOk is false. Primary goes down, all following queries fail with InvalidOperationException: Server instance <host>:<port> is no longer connected. (host/port match the ones of the gone server). Stack trace follows: MongoDB.Driver.DLL!MongoDB.Driver.MongoServerInstance.AcquireConnection(MongoDB.Driver.MongoDatabase database) Line 200 C# |
| Comment by Robert Stam [ 01/Aug/11 ] |
|
Comitted a fix for the deadlock. Added an additional lock (stateLock) to synchronize lower level state changes that happen in response to events on multiple threads. The remaining MongoServer operations are synchronized with serverLock. p.s. Still need to do some more testing on replica sets (specially failover) so there may be further change forthcoming. |
| Comment by Robert Stam [ 01/Aug/11 ] |
|
Fix had problems with deadlocks. More work needed. |
| Comment by Robert Stam [ 01/Aug/11 ] |
|
Added locking to MongoServerInstance to make it thread safe. Simplified locking in MongoServer by using just one lock (multiple locks were confusing and had the potential for deadlock). |
| Comment by Aristarkh Zagorodnikov [ 13/Jul/11 ] |
|
It also appears that MongoServer.AcquireConnection(MongoDatabase, MongoServerInstance) call hierarchy is not lock-protected at all (looks like it's used for cursors since they "bind" to a specific server instance), so its' calls to MongoServerInstance.AcquireConnection is also vulnerable to a race condition with MongoServerInstance.Disconnect calls. |
| Comment by Aristarkh Zagorodnikov [ 13/Jul/11 ] |
|
Yes, MongoServerInstance.Disconnect is called and the primary-before-the-step-down MongoServerInstance becomes disconnected, although MongoServer.GetServerInstance keeps returning it using the MongoServer.Primary property. Also, I would like to note that it appears that MongoServerInstance.Disconnect and MongoServerInstance.AcquireConnection can be called concurrently (there is no consistent locks around all of these calls), so this might lead to some kind of a nasty race condition. |