[SERVER-42627] Wait for index build to complete before destructing ServiceContextMongoDTest in ShardServerCatalogCacheLoaderTest unit tests Created: 05/Aug/19  Updated: 29/Oct/23  Resolved: 27/Aug/19

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 4.3 Required
Fix Version/s: 4.3.1

Type: Bug Priority: Major - P3
Reporter: Janna Golden Assignee: Janna Golden
Resolution: Fixed Votes: 0
Labels: sharding-wfbf-day
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
is related to SERVER-39454 Move createIndexes command fully onto... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Sharding 2019-09-09
Participants:
Linked BF Score: 20

 Description   

A number of the unit tests in shard_server_catalog_cache_loader_test.cpp call get() on the notification returned by ShardServerCatalogCacheLoader::getChunksSince() and then finish the test. We set this notification immediately after scheduling a task to persist metadata to config.cache.collections and config.cache.chunks.db.collname, meaning we do not actually wait for the metadata to be persisted before setting the notification. When persisting metadata to config.cache.chunks.db.collname, we create this collection and then run createIndex() on it through DBDirectClient. However, in the destructor for the unit test fixture (ServiceContextMongoDTest), we call DatabaseHolderImpl::closeAll() which invariants that we are not currently building any indexes.

This started happening frequently once storage switched to the new index builds implementation because they are now on a new thread pool. When we shutdown the shard server catalog cache loader, we call join on all the threads in its thread pool. Previously index builds were synchronous, so when tearing down one of these unit tests we would join the shard loader's threads which would include any index builds. Now, if one of the tasks on the shard loader's thread pool creates a collection and thus creates an index on that collection, this index build will be scheduled on a thread pool on the index builds coordinator. So, when joining the threads on the shard loader we aren't joining any index builds as well. When tearing down the unit tests, we call shutdown on the shard loader first (which won't stop any index builds) and then attempt to close dbs before shutting down the index builds coordinator. We should actually shutdown the index builds coordinator before attempting to close dbs since we invariant that there are no index builds ongoing when closing dbs.



 Comments   
Comment by Githook User [ 27/Aug/19 ]

Author:

{'name': 'Janna Golden', 'email': 'janna.golden@mongodb.com', 'username': 'jannaerin'}

Message: SERVER-42627 Shutdown IndexBuildsCoordinator before closing all dbs in ServiceContextMongoDTest fixture
Branch: master
https://github.com/mongodb/mongo/commit/278c646b57db863eea9f171821069e723f4b5f69

Generated at Thu Feb 08 05:00:59 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.