[SERVER-66002] Optimize `maxChunkSize` discovery/retrieval on config server Created: 27/Apr/22 Updated: 28/Apr/22 Resolved: 28/Apr/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Pierlauro Sciarelli | Assignee: | Allison Easton |
| Resolution: | Won't Do | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Sprint: | Sharding EMEA 2022-05-02 |
| Participants: |
| Description |
|
Following the introduction of the optional per-collection maxChunkSize in v5.1, when issuing moves the balancer has first to check whether a collection has a configured value or - if not - rely on the global value. This has led to a lot of inefficiency in discovering the max chunk size for a given collection since the catalog entry is continuously retrieved from disk when needing to balance (this happens for example when calling those functions). Moreover, with the modifications being done for the "no more auto splitter" project, the max chunk size is gaining more importance and needs to be retrieved/used in more parts in the code. Purpose of this ticket is to:
A further optimization - that can be offloaded in a follow-up ticket if too complex - would be to make the balancer configuration keep track in-memory of per-collection and global chunk sizes, with those value being updated via observers when their state changes on disk.
|
| Comments |
| Comment by Allison Easton [ 28/Apr/22 ] |
|
Since max chunk size is refreshed only at the beginning of the balancer round anyways and therefore is a best effort parameter, we will just continue to use the value from the balancer config refresh. |