[SERVER-30844] Reduce the number of forced metadata refreshes in the balancer code paths Created: 25/Aug/17  Updated: 07/Dec/22  Resolved: 07/Dec/22

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Dianna Hohensee (Inactive) Assignee: [DO NOT USE] Backlog - Sharding Team
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Sharding
Participants:

 Description   

We are forcing refresh for every chunk that the balancer tries to balance. This can be quite excessive. For example, the balancer decides to balance 10 chunks from the same collection in a single balancing round. It serially schedules each move, and then waits for all the responses – it will redundantly force refresh 10 times in a row on the same collection, once every time a chunk is scheduled. Scheduling rounds occur every 10 seconds.

Perhaps we could minimize refresh to once per collection we're trying to balance.

There are also several other places in the balancer code that we force refresh. They should all be audited and we should see if they can be minimized.

The mongos and shard servers are more reactive to changes, invalidating and refreshing as needed. The config server is rather more spurious, in general. Maybe we should reconsider making it more reactive, only invalidating it when a chunk metadata change occurs, rather than forcing refreshes wherever freshness is a concern.


Generated at Thu Feb 08 04:25:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.