[SERVER-41141] Remove DB X lock acquisition for secondary index builds Created: 14/May/19  Updated: 29/Oct/23  Resolved: 10/Jun/19

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.2.0-rc1, 4.3.1

Type: Bug Priority: Major - P3
Reporter: Louis Williams Assignee: Gregory Wlodarek
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Problem/Incident
Related
related to SERVER-41014 runCreateIndexesWithCoordinator does ... Closed
related to SERVER-42015 hybrid_index_with_updates.js hangs wh... Closed
is related to SERVER-39715 Do not drop index build temp tables o... Closed
is related to SERVER-39517 Only use Collection MODE_X for index ... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.2
Sprint: Execution Team 2019-06-03, Execution Team 2019-06-17
Participants:
Linked BF Score: 0

 Description   

The IndexBuildsCoordinator, which builds background indexes on secondaries, still takes a Database X lock, despite changes made on primaries to take a Collection X lock.

Secondaries should take the same locks as primaries, which could otherwise lead to unexpected deadlocks.



 Comments   
Comment by Githook User [ 10/Jun/19 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-41141 Remove DB X lock acquisition for secondary index builds
Branch: v4.2
https://github.com/mongodb/mongo/commit/dbb9a7c7acd0c3e97ff128c4eaa225b6567d2249

Comment by Githook User [ 10/Jun/19 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-41141 Remove DB X lock acquisition for secondary index builds
Branch: master
https://github.com/mongodb/mongo/commit/837d0766d57f2d3750f49277c7ecc345ef7d13fb

Comment by Gregory Wlodarek [ 07/Jun/19 ]

Unfortunately the codepath that rebuilds system indexes on startup for mobile uses the IndexBuildsCoordinator causing BF-13526 (hitting our isBackground invariant). Reverting for now until a solution for that is proposed.

Comment by Githook User [ 07/Jun/19 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: Revert "SERVER-41141 Remove DB X lock acquisition for secondary index builds"

This reverts commit fd2f851f6ce482064063286a6c763cf4dc2c4198.
Branch: master
https://github.com/mongodb/mongo/commit/41abf22263733d0dc96545b6043753d23f5851b9

Comment by Githook User [ 07/Jun/19 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: Revert "SERVER-41141 Remove DB X lock acquisition for secondary index builds"

This reverts commit 0c9d0e18109314efac6aa9f5352992cd4afc4d96.
Branch: v4.2
https://github.com/mongodb/mongo/commit/339cec66f0d98b6f9c84ca4ae8935ddc9f6cfaf0

Comment by Githook User [ 06/Jun/19 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-41141 Remove DB X lock acquisition for secondary index builds
Branch: v4.2
https://github.com/mongodb/mongo/commit/0c9d0e18109314efac6aa9f5352992cd4afc4d96

Comment by Githook User [ 06/Jun/19 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-41141 Release the mutex early in IndexBuildsManager::interruptIndexBuild() and IndexBuildsManager::abortIndexBuild() after getting a shared_ptr to the MultiIndexBlock to avoid a deadlock with index builds being aborted during server shutdown
Branch: v4.2
https://github.com/mongodb/mongo/commit/b05af50a62b900a659b502ca920c55f4b8ee4b3c

Comment by Githook User [ 06/Jun/19 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-41141 Remove DB X lock acquisition for secondary index builds
Branch: master
https://github.com/mongodb/mongo/commit/fd2f851f6ce482064063286a6c763cf4dc2c4198

Comment by Githook User [ 06/Jun/19 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-41141 Release the mutex early in IndexBuildsManager::interruptIndexBuild() and IndexBuildsManager::abortIndexBuild() after getting a shared_ptr to the MultiIndexBlock to avoid a deadlock with index builds being aborted during server shutdown
Branch: master
https://github.com/mongodb/mongo/commit/0a98a55f040de45dbf5e48968752c7053cd8acb9

Comment by Gregory Wlodarek [ 03/Jun/19 ]

benety.goh, you're right, I marked SERVER-39715 as depends on for this ticket.

Comment by Githook User [ 03/Jun/19 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: Revert "SERVER-41141 Remove DB X lock acquisition for secondary index builds"

This reverts commit 99185ab9685a51466e56ef0fd862aaf1318a79d6.
Branch: master
https://github.com/mongodb/mongo/commit/16a18a5370f276ca5ec91bfc13b2738390b319e8

Comment by Githook User [ 03/Jun/19 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-41141 Remove DB X lock acquisition for secondary index builds
Branch: master
https://github.com/mongodb/mongo/commit/99185ab9685a51466e56ef0fd862aaf1318a79d6

Comment by Benety Goh [ 01/Jun/19 ]

One of the proposed solutions for this ticket, currently in review, downgrades the exclusive lock in IndexBuildsCoordinator::_runIndexBuildInner() to an intent lock. This may affect the removal of temporary record stores created by hybrid index builds during a clean shutdown. This behavior is best characterized by the JS test jstests/noPassthrough/indexbg_shutdown.js.

SERVER-39715 describes a possible solution to this issue.

Generated at Thu Feb 08 04:56:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.