[SERVER-60485] Tasks which run on the MigrationUtilExecutor must not wait for shutdown to complete Created: 06/Oct/21  Updated: 29/Oct/23  Resolved: 07/Apr/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 5.0.3, 4.4.9, 5.1.0-rc0, 5.3.0-rc4
Fix Version/s: 5.3.2, 5.0.8, 6.0.0-rc0, 4.4.20

Type: Bug Priority: Major - P3
Reporter: Kaloian Manassiev Assignee: Tommaso Tocci
Resolution: Fixed Votes: 0
Labels: sharding-wfbf-sprint, shardingemea-qw
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Problem/Incident
is caused by SERVER-26145 Shutdown during move chunk commit fas... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.3, v5.0, v4.4
Sprint: Sharding EMEA 2021-11-01, Sharding EMEA 2021-12-27, Sharding EMEA 2022-01-10, Sharding EMEA 2022-01-24, Sharding EMEA 2022-03-21, Sharding EMEA 2022-05-02
Participants:
Linked BF Score: 33
Story Points: 3

 Description   

All the blocking range deletion recovery work happens on the MigrationUtilExecutor and eventually calls into retryIdempotentWorkAsPrimaryUntilSuccessOrStepdown, which is a strictly blocking call that relies on globalInShutdownDeprecated/waitForShutdown.

Given that we join the MigrationUtilExecutor on shutdown, this can result in a deadlock where the shutdown thread is waiting for the executor to complete and the executor is waiting on shutdown to complete.



 Comments   
Comment by Githook User [ 28/Mar/23 ]

Author:

{'name': 'Tommaso Tocci', 'email': 'tommaso.tocci@mongodb.com', 'username': 'toto-dev'}

Message: SERVER-60485 Tasks which run on the MigrationUtilExecutor must not wait for shutdown to complete

(cherry picked from commit f68132598722199fec72468599aef83d507ab213)
(cherry picked from commit a8450a4f4ed7b7b47957dbaa0d6393fc4f0fd35a)
Branch: v4.4
https://github.com/mongodb/mongo/commit/1edfde2eb7b6404a6c52da18afb9562d5da319a2

Comment by Githook User [ 07/Apr/22 ]

Author:

{'name': 'Tommaso Tocci', 'email': 'tommaso.tocci@mongodb.com', 'username': 'toto-dev'}

Message: SERVER-60485 Tasks which run on the MigrationUtilExecutor must not wait for shutdown to complete

(cherry picked from commit f68132598722199fec72468599aef83d507ab213)
Branch: v5.3
https://github.com/mongodb/mongo/commit/f3ecbd757504b42891bf51f5a0ff90947a07f235

Comment by Githook User [ 07/Apr/22 ]

Author:

{'name': 'Tommaso Tocci', 'email': 'tommaso.tocci@mongodb.com', 'username': 'toto-dev'}

Message: SERVER-60485 Tasks which run on the MigrationUtilExecutor must not wait for shutdown to complete

(cherry picked from commit f68132598722199fec72468599aef83d507ab213)
Branch: v5.0
https://github.com/mongodb/mongo/commit/a8450a4f4ed7b7b47957dbaa0d6393fc4f0fd35a

Comment by Githook User [ 07/Apr/22 ]

Author:

{'name': 'Tommaso Tocci', 'email': 'tommaso.tocci@mongodb.com', 'username': 'toto-dev'}

Message: SERVER-60485 Tasks which run on the MigrationUtilExecutor must not wait for shutdown to complete
Branch: master
https://github.com/mongodb/mongo/commit/f68132598722199fec72468599aef83d507ab213

Generated at Thu Feb 08 05:49:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.