[SERVER-59888] Deadlock between BucketCatalog::_statesMutex and BucketCatalog::_idleMutex when expiring idle buckets Created: 10/Sep/21 Updated: 29/Oct/23 Resolved: 13/Sep/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 5.0.3, 5.1.0-rc0 |
| Type: | Bug | Priority: | Critical - P2 |
| Reporter: | Gregory Noma | Assignee: | Gregory Noma |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||
| Issue Links: |
|
||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||
| Operating System: | ALL | ||||||||||||
| Backport Requested: |
v5.0
|
||||||||||||
| Sprint: | Execution Team 2021-09-20 | ||||||||||||
| Participants: | |||||||||||||
| Case: | (copied to CRM) | ||||||||||||
| Description |
|
Consider two concurrent time-series inserts: one which inserts into an existing bucket, and one which expires an idle bucket in order to allocate a new one. The former acquires BucketCatalog::_statesMutex inside of BucketCatalog::BucketAccess::_confirmStateForAcquiredBucket and then later acquires BucketCatalog::_idleMutex inside of BucketCatalog::_markBucketNotIdle. The latter acquires BucketCatalog::_idleMutex inside of BucketCatalog::_expireIdleBuckets and then later acquires BucketCatalog::_statesMutex inside of BucketCatalog::_removeBucket. Deadlock. The full BucketCatalog stack traces to reach this deadlock are:
and
Attached is a patch which reproduces the deadlock. |
| Comments |
| Comment by Githook User [ 14/Sep/21 ] |
|
Author: {'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}Message: (cherry picked from commit 6bd9072c982bec88a61573ec723bce711a6c54a1) |
| Comment by Githook User [ 13/Sep/21 ] |
|
Author: {'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}Message: |