[SERVER-29027] Allow collections in the config database to be sharded Created: 01/May/17 Updated: 30/Oct/23 Resolved: 28/Sep/17 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | None |
| Fix Version/s: | 3.6.0-rc0 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Samantha Ritter (Inactive) | Assignee: | Samantha Ritter (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Backwards Compatibility: | Minor Change | ||||||||||||||||||||||||
| Sprint: | Platforms 2017-10-02 | ||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Description |
|
The logical sessions project requires the ability to create a new sharded system collection in the config database. This is currently not possible. |
| Comments |
| Comment by Githook User [ 28/Sep/17 ] |
|
Author: {'email': 'samantha.ritter@10gen.com', 'name': 'samantharitter', 'username': 'samantharitter'}Message: |
| Comment by Andy Schwerin [ 11/Aug/17 ] |
|
Oh, great. |
| Comment by Kaloian Manassiev [ 11/Aug/17 ] |
|
The 'config' shard is always present on the shard registry, so this shouldn't be a problem. For the config and admin databases not having entry in the catalog cache, we always return a hardcoded result, so these databases magically exist as well. |
| Comment by Andy Schwerin [ 11/Aug/17 ] |
|
kaloian.manassiev, isn't the hard part that the config server itself doesn't appear in `config.shards`? And that the `config` database doesn't have an entry in `config.dbs` or whatever it's called? |
| Comment by Kaloian Manassiev [ 09/Aug/17 ] |
|
I propose that we implement this by marking the config.system.sessions collection as sharded in the metadata, like we do for any other sharded collection, but we also hardcode knowledge in the balancer policy that no chunks for any sharded collection residing under config or admin may be moved to the config shard. This will ensure that both auto-balancing and moveChunk will not violate this policy and will also make this backwards-compatible with 3.4. If users manually modify the routing information, the server will start returning errors for any attempt to cache this routing information, which would only make creating sessions impossible, but the remaining collections will be able to be accessed. TL;DR: With the ability of shards to store copy of the routing metadata for the purposes of consistent secondary reads, it is currently not possible to treat the config server as just another shard. This is because it holds the authoritative copy of the routing information in a format which differs from that of the shards. It might be possible to do safe secondary reads by tweaking the ConfigServerCatalogCacheLoader, however for the purposes of being able to shard the config.system.sessions collection in time for the 3.6 release we can do this intermediate solution. This brings us one step closer to treating the config server as a shard. cc schwerin, alyson.cabral about whether there are any objections to implementing this behaviour. I estimate that implementing this would take roughly about 1-2 weeks for a single developer working on it 100% of the time, including writing targeted JS tests. |