[DOCS-16132] Investigate changes in SERVER-77018: Deadlock between dbStats and 2 index builds Created: 17/May/23  Updated: 13/Nov/23  Resolved: 02/Aug/23

Status: Closed
Project: Documentation
Component/s: manual, Server
Affects Version/s: None
Fix Version/s: 7.1.0-rc0, 6.0.7, 6.3.2, 5.0.19, 7.0.0-rc2, Server_Docs_20231030, Server_Docs_20231106, Server_Docs_20231105, Server_Docs_20231113

Type: Task Priority: Major - P3
Reporter: Backlog - Core Eng Program Management Team Assignee: Corry Root
Resolution: Done Votes: 0
Labels: None
Σ Remaining Estimate: Not Specified Remaining Estimate: Not Specified
Σ Time Spent: Not Specified Time Spent: Not Specified
Σ Original Estimate: Not Specified Original Estimate: Not Specified

Issue Links:
Documented
documents SERVER-77018 Deadlock between dbStats and 2 index ... Closed
Related
related to DOCS-16152 [BACKPORT] [v7.0] Deadlock between db... Closed
is related to DOCS-16162 [BACKPORT] [v6.3] Deadlock between db... Closed
Sub-Tasks:
Key
Summary
Type
Status
Assignee
DOCS-16296 [Server] Backport DOCS-16132 to v6.3. Backport Sub-Task Closed Corry Root  
DOCS-16297 [Server] Backport DOCS-16132 to v6.0. Backport Sub-Task Closed Corry Root  
DOCS-16298 [Server] Backport DOCS-16132 to v5.0 Backport Sub-Task Closed Corry Root  
Participants:
Days since reply: 34 weeks, 1 day ago
Story Points: 2

 Description   
Original Downstream Change Summary

In-progress index builds are no longer accounted for 'indexFreeStorageSize' when running dbStats.

Description of Linked Ticket

If an on-going index build yields its locks after initiating a bulk insert (which is initialized here), it still holds onto the write lock on the index table at the WiredTiger level. If a dbStats command comes in, it will take collection level MODE_IS lock and attempt to acquire a read_lock for the ident the index build is currently writing to (but cannot since IndexBuild_1 holds the exclusive lock on that ident). (In (collection_impl.cpp) we iterate through the unfinished indexes and that is how we can see the in-progress index table).
  

The problem arises when another operation comes in and prevents IndexBuild_1 from re-acquiring its lock, like another index build that enqueues a collection MODE_X lock. These events can produce a deadlock in the system represented by:

dbStats IndexBuild_0 IndexBuild_1
[Global, DB, Coll]- MODE_IS [Global, DB] - MODE_IX  [Global, DB, Coll] - MODE_IX 
    yields MDB level locks
- holds write lock on table:index-X
blocks IndexBuild_1
- waiting on read lock of table:index-X
- holds coll lock - MODE_IS 
   
 
  • waiting for MODE_X coll lock
 
   
  • waiting to reacquire locks

 

Original explanation by Suganthi here



 Comments   
Comment by Sarah Olson [ 13/Jun/23 ]

Closing out two additional backport requests. Backports to all supported branches will be done as part of this ticket.

https://jira.mongodb.org/browse/DOCS-16199

https://jira.mongodb.org/browse/DOCS-16198

 

 

Generated at Thu Feb 08 08:14:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.