[SERVER-16133] Deadlock detected with oplog and multikey index inserts Created: 13/Nov/14 Updated: 10/Dec/14 Resolved: 14/Nov/14 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Concurrency, Replication |
| Affects Version/s: | 2.8.0-rc0 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | David Percy | Assignee: | Kaloian Manassiev |
| Resolution: | Done | Votes: | 0 |
| Labels: | 28qa | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Operating System: | ALL |
| Participants: |
| Description |
|
We ran a workload of 30 client threads doing inserts into a collection with a multikey index. Each inserted document had a 10-element array as the multikey-indexed field. We're seeing many "Deadlock found..." and "WriteConflictException while doing oplog application" log messages. Neither of these messages shows up when we don't use replication or when we remove the multikey index. Log excerpt (full log attached):
|
| Comments |
| Comment by Githook User [ 24/Nov/14 ] |
|
Author: {u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}Message: Revert " This reverts commit 58e49b2726807b7e7c2f0309607ffa454befbff6. |
| Comment by Githook User [ 24/Nov/14 ] |
|
Author: {u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}Message: This avoids the need to do deadlock detection and as a nice side effect Also, is a matter of correctness, we should not be setting the _isMultiKey |
| Comment by Kaloian Manassiev [ 14/Nov/14 ] |
|
This is an informational message only. It indicates that a deadlock was discovered and handled successfully by the server. What happens in this particular case is that there are multiple concurrent inserts for the first time in a multi-key index, which need to set this property. The property may only be set under an exclusive collection lock, so all these inserts try to upgrade from IX to X. This is a deadlock since nobody can succeed, so they all yield the lock and one tries again with the X-lock directly. The message is from the retry and is expected occasionally much like WriteConflict is. We are working on improving the message though so it is more descriptive. |