[SERVER-20854] Don't refresh database and collection information from config server while holding a mutex Created: 09/Oct/15  Updated: 28/Aug/18  Resolved: 04/Apr/17

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 3.4.4, 3.5.6

Type: Bug Priority: Major - P3
Reporter: Spencer Brody (Inactive) Assignee: Kaloian Manassiev
Resolution: Done Votes: 9
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
depends on SERVER-22611 ChunkManager refresh can occasionally... Closed
depends on SERVER-27321 Rename ScopedChunkManager::getExistin... Closed
depends on SERVER-27480 Get rid of ConfigServer class Closed
depends on SERVER-27526 Get rid of all non-metadata usages of... Closed
depends on SERVER-27527 Remove explicit shardingEnabled check... Closed
depends on SERVER-27736 DBConfig should not store unsharded c... Closed
depends on SERVER-27809 Move split logic outside of the Chunk... Closed
depends on SERVER-28030 Move 'initial chunks' writes out of C... Closed
depends on SERVER-28098 Get rid of DBConfig::getAllShardIds Closed
depends on SERVER-28106 Cleanup the contract of ChunkManager:... Closed
depends on SERVER-28111 Move parallel.h/parallel.cpp to be un... Closed
depends on SERVER-27364 Remove DBConfig::setPrimary Closed
depends on SERVER-27366 Remove DBConfig::enableSharding Closed
depends on SERVER-27381 Remove DBConfig::dropDatabase Closed
depends on SERVER-27382 Remove usages of DBConfig::reload Closed
depends on SERVER-27419 Move cluster 'drop' command to a sepa... Closed
depends on SERVER-27467 Remove double loading of the sharding... Closed
depends on SERVER-27473 Remove writes from DBConfig Closed
depends on SERVER-27681 Better concurrency control on shard m... Closed
depends on SERVER-28155 Cleanup sharding library dependencies Closed
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v3.4
Sprint: Sharding 2016-12-12, Sharding 2017-01-02, Sharding 2017-02-13, Sharding 2017-03-06, Sharding 2017-03-27, Sharding 2017-04-17
Participants:
Case:

 Description   

CatalogCache::getDatabase loads database metadata under the CatalogCache's mutex, blocking all operations while the config server query completes. This alone is pretty bad, but at least the CatalogCache shouldn't miss too often (CatalogCache caches database objects, database creation should be relatively infrequent).

Even worse is the reloading of the ChunkManager, which loads the chunk ranges under the DBConfig's mutex. This means that while we are loading the chunk ranges for a collection after a shard version error, all CRUD operations on all (sharded or unsharded) collections in that database will block until the chunk reload is finished.



 Comments   
Comment by Githook User [ 13/Apr/17 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-20854 Refresh collection metadata on a separate thread

(cherry picked from commit 0885a0adbb813727cc5a2083224ac0d89763c276)
Branch: v3.4
https://github.com/mongodb/mongo/commit/48d1e1bd6f56f3c20c018332fabafc147bfafc3b

Comment by Githook User [ 11/Apr/17 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-20854 Pull the diff tracker's query generation logic outside of the template

(cherry picked from commit 0c7ab1cdd8c6a1cde4c5b2ccb1feef4300d29708)
Branch: v3.4
https://github.com/mongodb/mongo/commit/ed67f8a91fe22f8667d64f57f9fad1e404ba3f05

Comment by Githook User [ 04/Apr/17 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-20854 Refresh collection metadata on a separate thread
Branch: master
https://github.com/mongodb/mongo/commit/0885a0adbb813727cc5a2083224ac0d89763c276

Comment by Githook User [ 02/Mar/17 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-20854 Pull the diff tracker's query generation logic outside of the template
Branch: master
https://github.com/mongodb/mongo/commit/0c7ab1cdd8c6a1cde4c5b2ccb1feef4300d29708

Generated at Thu Feb 08 03:55:30 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.