[SERVER-38171] Admin commmand to load the metadata in the sharded cluster Created: 16/Nov/18  Updated: 08/Jan/24  Resolved: 03/Aug/20

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Linda Qin Assignee: Backlog - Service Architecture
Resolution: Won't Fix Votes: 0
Labels: PM-1211, mongos-drain-mode-fallout
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
related to SERVER-17124 Ability to start a node in maintenanc... Backlog
related to SERVER-31156 Admin command to update the chunk met... Closed
Assigned Teams:
Service Arch
Participants:
Case:

 Description   

Currently the mongos/mongod load the metadata lazily - only when they access a collection, they will load the chunks metadata for that specific collection.

For large collections with tens/hundreds of thousands of chunks, loading the metadata may take some time. There are some situations that loading the metadata proactively may help. For example, after a mongos is restarted and accepts the application traffic, the operations would be pending until the chunk metadata for the collection (s) is loaded. This would increase the latency for the operations.

It would be nice if we can provide the ability (e.g. an admin command) to load the metadata (for all the collections), so that it can be run after a mongos is started, and before this mongos is used for application traffic.



 Comments   
Comment by Mira Carey [ 26/Feb/20 ]

For an update on this ticket, work to allow prewarming of connection pools and preloading of the routing table occured as part of PM-1626 for mongos.

SERVER-44152 - Pre-warm connection pools in mongos
SERVER-43985 - Make mongos pre-cache the routing table on startup

I see that this ticket also mentions mongod, so not closing as a dupe, but wanted to point out for anyone who's watching that some progress has been made on this issue.

Also interested if there's still an appetite after that change

Comment by Kaloian Manassiev [ 08/May/19 ]

This is not strictly 4.1 Required, but I think we should decide whether to do it as a Quick Win instead of waiting for the complete feature to land.

Comment by Gregory McKeon (Inactive) [ 06/May/19 ]

mira.carey@mongodb.com should we move this to the is this 4.1 required?

Comment by Mira Carey [ 16/Nov/18 ]

I think a command for this isn't quite what you want. Outside of customers explicitly load balancing mongos' (something that very few do today), there's no good way to make a mongos available after a client application has been started. Because of that, an admin command isn't fully useful (as you have to start accepting traffic to run the command, at which point you're already getting client traffic that could have been faster if run somewhere else).

Would you be satisfied with a start up set parameter that forced a greedy load of the metadata? From there we could postpone accepting connections until after the metadata was loaded.

Generated at Thu Feb 08 04:48:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.