[SERVER-43985] Make mongos pre-cache the routing table on startup Created: 14/Oct/19 Updated: 16/Mar/23 Resolved: 28/Oct/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | None |
| Fix Version/s: | 4.3.1, 4.2.18 |
| Type: | New Feature | Priority: | Major - P3 |
| Reporter: | Esha Maharishi (Inactive) | Assignee: | Anton Oyung |
| Resolution: | Done | Votes: | 0 |
| Labels: | jason-carey-upgrade | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Backport Requested: |
v4.2
|
||||||||||||||||
| Sprint: | Sharding 2019-10-21, Sharding 2019-11-04 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||
| Description |
Problem statementThere are several types of performance problems due to the routing metadata. One is the predictable, high perf impact of mongos fetching the entire routing table from the config server at cold start-up. ProposalMake mongos preload the routing table for all collections on startup. 1) Use ShardingCatalogClient::getAllDBs to get a vector of all the databases. 2) For each database in the vector, use CatalogCache::getAllShardedCollectionsForDb to get all its sharded collections. 3) For each sharded collection, use CatalogCache::getShardedCollectionRoutingInfoWithRefresh to load the routing info for that collection. This logic can be placed here in sharding initialization on mongos, which occurs synchronously during mongos startup. Test that the logic works by starting up a mongos and asserting on CatalogCache::Stats::countFullRefreshesStarted, which is returned by serverStatus. |
| Comments |
| Comment by Githook User [ 07/Oct/21 ] |
|
Author: {'name': 'Anton Oyung', 'email': 'anton.oyung@mongodb.com', 'username': 'AntonOyung'}Message: |
| Comment by Githook User [ 28/Oct/19 ] |
|
Author: {'name': 'Anton Oyung', 'username': 'AntonOyung', 'email': 'anton.oyung@mongodb.com'}Message: “ |