[GODRIVER-1234] Deadlock on 1.1.0 release Created: 13/Aug/19 Updated: 28/Oct/23 Resolved: 30/Aug/19 |
|
| Status: | Closed |
| Project: | Go Driver |
| Component/s: | Core API |
| Affects Version/s: | None |
| Fix Version/s: | 1.2.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | David Bartley | Assignee: | Isabella Siu (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Description |
|
We recently attempted to upgrade to the 1.1.0 release and one of our tests caught a deadlock in Topology.Connect.
It looks like Topology.Connect takes out a lock on Topology.serversLock and calls Topology.addServer, but then Topology.addServer is recursively called, and calls Topology.apply, which also takes a lock on Topology.serversLock. |
| Comments |
| Comment by Githook User [ 03/Sep/19 ] | |||||||||||||||||||||||||||
|
Author: {'username': 'iwysiu', 'email': 'isabella.siu@10gen.com', 'name': 'iwysiu'}Message: Change-Id: I478a49ae07c8106c88db520f42b97d27c1b2448d | |||||||||||||||||||||||||||
| Comment by Githook User [ 30/Aug/19 ] | |||||||||||||||||||||||||||
|
Author: {'name': 'iwysiu', 'username': 'iwysiu', 'email': 'isabella.siu@10gen.com'}Message: Change-Id: I478a49ae07c8106c88db520f42b97d27c1b2448d | |||||||||||||||||||||||||||
| Comment by Isabella Siu (Inactive) [ 28/Aug/19 ] | |||||||||||||||||||||||||||
|
code review url: https://review.gerrithub.io/c/mongodb/mongo-go-driver/+/466625 | |||||||||||||||||||||||||||
| Comment by David Bartley [ 18/Aug/19 ] | |||||||||||||||||||||||||||
|
Er, more accurately, that commit (94c698b5b1ee056e5d9995e7d55df59ba62213be) changed things so that MinPoolSize was actually honoured; I believe prior to that commit it was just ignored. | |||||||||||||||||||||||||||
| Comment by David Bartley [ 18/Aug/19 ] | |||||||||||||||||||||||||||
|
Ah, figured it out. This is caused by the new MinPoolSize feature, which was added in the commit I mentioned. That forces connections to be created as part of the Connect call (in resourcePool.Maintain). Here's a small repro:
I'm not sure what the best fix is, but it still seems like the real issue is the coarseness of the serversLock. | |||||||||||||||||||||||||||
| Comment by David Bartley [ 14/Aug/19 ] | |||||||||||||||||||||||||||
|
I haven't tested it outside of tests, but it seems to reliably fail a number of tests that simply start a standalone or replset mongod (and then create and connect to a Topology). I've bisected the bad commit to 94c698b5b1ee056e5d9995e7d55df59ba62213be, though it's not clear to me what in particular from that commit would have started triggering this deadlock; it looks like it's always been possible to trigger it, well before that commit. | |||||||||||||||||||||||||||
| Comment by Kristofer Brandow (Inactive) [ 14/Aug/19 ] | |||||||||||||||||||||||||||
|
Hi bartle, Is this happening all the time when you try to connect the driver or is it only in one specific test? --Kris |