[SERVER-43985] Make mongos pre-cache the routing table on startup Created: 14/Oct/19  Updated: 16/Mar/23  Resolved: 28/Oct/19

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 4.3.1, 4.2.18

Type: New Feature Priority: Major - P3
Reporter: Esha Maharishi (Inactive) Assignee: Anton Oyung
Resolution: Done Votes: 0
Labels: jason-carey-upgrade
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Related
related to SERVER-47169 Sharding initialization contacts conf... Closed
related to SERVER-62036 Failures in preCacheMongosRoutingInfo... Closed
Backport Requested:
v4.2
Sprint: Sharding 2019-10-21, Sharding 2019-11-04
Participants:
Case:

 Description   

Problem statement

There are several types of performance problems due to the routing metadata. One is the predictable, high perf impact of mongos fetching the entire routing table from the config server at cold start-up.

Proposal

Make mongos preload the routing table for all collections on startup.

1) Use ShardingCatalogClient::getAllDBs to get a vector of all the databases.

2) For each database in the vector, use CatalogCache::getAllShardedCollectionsForDb  to get all its sharded collections.

3) For each sharded collection, use CatalogCache::getShardedCollectionRoutingInfoWithRefresh to load the routing info for that collection.

This logic can be placed  here in sharding initialization on mongos, which occurs synchronously during mongos startup.

Test that the logic works by starting up a mongos and asserting on CatalogCache::Stats::countFullRefreshesStarted, which is returned by serverStatus.



 Comments   
Comment by Githook User [ 07/Oct/21 ]

Author:

{'name': 'Anton Oyung', 'email': 'anton.oyung@mongodb.com', 'username': 'AntonOyung'}

Message: SERVER-43985 Make mongos pre-cache the routing table on startup
Branch: v4.2
https://github.com/mongodb/mongo/commit/1fd7913b67d45874f1fde8f6297f4adf618e3d34

Comment by Githook User [ 28/Oct/19 ]

Author:

{'name': 'Anton Oyung', 'username': 'AntonOyung', 'email': 'anton.oyung@mongodb.com'}

Message: “SERVER-43985
Branch: master
https://github.com/mongodb/mongo/commit/349d3c84887940964083dfac2d9ffd36122771b5

Generated at Thu Feb 08 05:04:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.