[SERVER-82216] Make a more even distribution of chunks through shards for initial split policies with zones Created: 16/Oct/23 Updated: 26/Oct/23 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Pol Pinol | Assignee: | Backlog - Catalog and Routing |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | car-qw, oldshardingemea | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Catalog and Routing
|
||||||||
| Participants: | |||||||||
| Story Points: | 2 | ||||||||
| Description |
|
Choosing the correct strategy to initialize chunks to the shards is important to have an even distribution, i.e. all shards have the same amount of chunks. Today’s createFirstChunks for initial split policies with zones will first create one (or some) chunk per zone and fill the gaps not contained in the zone’s domain. The current implementation distributes the chunks already created starting from the MinKey to the MaxKey, using a round-robin strategy. Because of zones can enforce chunks to be placed into specific shards, we could end up having some shards with more chunks than others. An example of that situation could be: Cluster with three shards (Shard0, Shard1, Shard2):
Chunk Distribution:
I propose improving this algorithm with first distributing all chunks associated with a zone, and then fill the gaps according to the number of chunks per shard. |