[SERVER-46704] Two phase index build can violate locking ordering and can lead to deadlocks. Created: 09/Mar/20 Updated: 29/Oct/23 Resolved: 19/Mar/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | 4.4.0-rc0, 4.7.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Suganthi Mani | Assignee: | Suganthi Mani |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||||||||||||||||||||
| Backport Requested: |
v4.4
|
||||||||||||||||||||||||||||||||||||||||||||
| Sprint: | Execution Team 2020-03-09, Execution Team 2020-03-23 | ||||||||||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||||||||||
| Linked BF Score: | 49 | ||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
Currently, IndexBuildsCoordinatorMongod::voteCommitIndexBuild() violates the lock ordering, i.e., it tries to acquire RSTL lock in mode IX with ReplIndexBuildState::mutex held. As a result, it can deadlock with stepup code path (ReplicationCoordinatorImpl::signalDrainComplete), as it acquires RSTL lock in X mode first, and then tries to send abort or commit signal to index build by holding ReplIndexBuildState::mutex. Note: 2) We are acquiring collection lock in stronger mode (mode X) in order to commit or abort. As, a result, this can lead to deadlocks involving prepared transactions, stepdown and indexBuildsCoordinator. ( 3) Currently IndexBuildsCoordinatorMongod::_waitForNextIndexBuildAction() holds RSTL only for the while loop scope. As a result, the primary check that we are doing at this line, can no longer be valid. (
UPDATE: This ticket won't address the 3 additional issues. And it will be addressed separately. |
| Comments |
| Comment by Githook User [ 19/Mar/20 ] |
|
Author: {'email': 'suganthi.mani@mongodb.com', 'name': 'Suganthi Mani', 'username': 'smani87'}Message: (cherry picked from commit 1d7f1c5d9e482b7508a1879eda9835cd1ea2c185) |
| Comment by Githook User [ 19/Mar/20 ] |
|
Author: {'email': 'suganthi.mani@mongodb.com', 'name': 'Suganthi Mani', 'username': 'smani87'}Message: |