[SERVER-71191] Deadlock between index build setup, prepared transaction, and stepdown Created: 08/Nov/22  Updated: 29/Oct/23  Resolved: 11/Nov/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.1.1, 4.4.19, 5.0.15, 6.0.4, 6.2.0-rc0

Type: Bug Priority: Major - P3
Reporter: Gregory Noma Assignee: Gregory Noma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
related to SERVER-42621 3 way deadlock can happen between hyb... Closed
related to SERVER-44722 3 way deadlock can happen between hyb... Closed
is related to SERVER-71198 Assert that unkillable operations tha... Backlog
is related to SERVER-71288 Consolidate index build RSTL unlock/r... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.1, v6.0, v5.0, v4.4
Sprint: Execution Team 2022-11-14, Execution Team 2022-11-28
Participants:
Linked BF Score: 135

 Description   

When setting up an index build, we take a collection MODE_X lock. This can lead to a deadlock if there is also a replication state transition along with a prepared transaction. The index builder holds the RSTL in MODE_IX with the state transition waiting to take the RSTL in MODE_X. The index builder can't acquire the collection lock however because a prepared transaction is holding the collection lock in MODE_IX. In particular this is an issue because the index builder is purposely not interrupted for the state transition. See SERVER-44722 describing a similar issue. We should be able to fix this by unlocking the RSTL and retrying with a timeout like we do during index build commit.



 Comments   
Comment by Githook User [ 28/Nov/22 ]

Author:

{'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}

Message: SERVER-71191 Unlock and relock RSTL during index build setup

(cherry picked from commit f1203e555321e326b0621479bcc6b607c5854391)
Branch: v4.4
https://github.com/mongodb/mongo/commit/aaa13e066cbddbfbdc1debc41a8fe903100bf9e9

Comment by Githook User [ 18/Nov/22 ]

Author:

{'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}

Message: SERVER-71191 Unlock and relock RSTL during index build setup

(cherry picked from commit f1203e555321e326b0621479bcc6b607c5854391)
Branch: v5.0
https://github.com/mongodb/mongo/commit/a5d69928f116c7cfe1f9441f0bc0c13a82ae2442

Comment by Githook User [ 16/Nov/22 ]

Author:

{'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}

Message: SERVER-71191 Unlock and relock RSTL during index build setup

(cherry picked from commit f1203e555321e326b0621479bcc6b607c5854391)
Branch: v6.0
https://github.com/mongodb/mongo/commit/94a97b66b41f43f6de0fe1adaef3a06ac75d3807

Comment by Githook User [ 16/Nov/22 ]

Author:

{'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}

Message: SERVER-71191 Unlock and relock RSTL during index build setup

(cherry picked from commit f1203e555321e326b0621479bcc6b607c5854391)
Branch: v6.1
https://github.com/mongodb/mongo/commit/dc08ee3d6e52cbea9a643571e214b8943a7d60ed

Comment by Githook User [ 11/Nov/22 ]

Author:

{'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}

Message: SERVER-71191 Unlock and relock RSTL during index build setup
Branch: master
https://github.com/mongodb/mongo/commit/f1203e555321e326b0621479bcc6b607c5854391

Generated at Thu Feb 08 06:18:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.