[SERVER-38171] Admin commmand to load the metadata in the sharded cluster Created: 16/Nov/18 Updated: 08/Jan/24 Resolved: 03/Aug/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Linda Qin | Assignee: | Backlog - Service Architecture |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | PM-1211, mongos-drain-mode-fallout | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Assigned Teams: |
Service Arch
|
||||||||||||||||
| Participants: | |||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||
| Description |
|
Currently the mongos/mongod load the metadata lazily - only when they access a collection, they will load the chunks metadata for that specific collection. For large collections with tens/hundreds of thousands of chunks, loading the metadata may take some time. There are some situations that loading the metadata proactively may help. For example, after a mongos is restarted and accepts the application traffic, the operations would be pending until the chunk metadata for the collection (s) is loaded. This would increase the latency for the operations. It would be nice if we can provide the ability (e.g. an admin command) to load the metadata (for all the collections), so that it can be run after a mongos is started, and before this mongos is used for application traffic. |
| Comments |
| Comment by Mira Carey [ 26/Feb/20 ] |
|
For an update on this ticket, work to allow prewarming of connection pools and preloading of the routing table occured as part of PM-1626 for mongos.
I see that this ticket also mentions mongod, so not closing as a dupe, but wanted to point out for anyone who's watching that some progress has been made on this issue. Also interested if there's still an appetite after that change |
| Comment by Kaloian Manassiev [ 08/May/19 ] |
|
This is not strictly 4.1 Required, but I think we should decide whether to do it as a Quick Win instead of waiting for the complete feature to land. |
| Comment by Gregory McKeon (Inactive) [ 06/May/19 ] |
|
mira.carey@mongodb.com should we move this to the is this 4.1 required? |
| Comment by Mira Carey [ 16/Nov/18 ] |
|
I think a command for this isn't quite what you want. Outside of customers explicitly load balancing mongos' (something that very few do today), there's no good way to make a mongos available after a client application has been started. Because of that, an admin command isn't fully useful (as you have to start accepting traffic to run the command, at which point you're already getting client traffic that could have been faster if run somewhere else). Would you be satisfied with a start up set parameter that forced a greedy load of the metadata? From there we could postpone accepting connections until after the metadata was loaded. |