[SERVER-66002] Optimize `maxChunkSize` discovery/retrieval on config server Created: 27/Apr/22  Updated: 28/Apr/22  Resolved: 28/Apr/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Pierlauro Sciarelli Assignee: Allison Easton
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Sprint: Sharding EMEA 2022-05-02
Participants:

 Description   

Following the introduction of the optional per-collection maxChunkSize in v5.1, when issuing moves the balancer has first to check whether a collection has a configured value or - if not - rely on the global value.

This has led to a lot of inefficiency in discovering the max chunk size for a given collection since the catalog entry is continuously retrieved from disk when needing to balance (this happens for example when calling those functions).

Moreover, with the modifications being done for the "no more auto splitter" project, the max chunk size is gaining more importance and needs to be retrieved/used in more parts in the code.

Purpose of this ticket is to:

  • Allow to retrieve per-collection maxChunkSize from the balancer configuration rather than populating the code with plenty of calls to the catalog client. This can be as simple of offering a method that is retrieving the value from the catalog without the caller having to manually do it.

A further optimization - that can be offloaded in a follow-up ticket if too complex - would be to make the balancer configuration keep track in-memory of per-collection and global chunk sizes, with those value being updated via observers when their state changes on disk.

 



 Comments   
Comment by Allison Easton [ 28/Apr/22 ]

Since max chunk size is refreshed only at the beginning of the balancer round anyways and therefore is a best effort parameter, we will just continue to use the value from the balancer config refresh.

Generated at Thu Feb 08 06:04:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.