[SERVER-30844] Reduce the number of forced metadata refreshes in the balancer code paths Created: 25/Aug/17 Updated: 07/Dec/22 Resolved: 07/Dec/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Dianna Hohensee (Inactive) | Assignee: | [DO NOT USE] Backlog - Sharding Team |
| Resolution: | Won't Do | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Assigned Teams: |
Sharding
|
| Participants: |
| Description |
|
We are forcing refresh for every chunk that the balancer tries to balance. This can be quite excessive. For example, the balancer decides to balance 10 chunks from the same collection in a single balancing round. It serially schedules each move, and then waits for all the responses – it will redundantly force refresh 10 times in a row on the same collection, once every time a chunk is scheduled. Scheduling rounds occur every 10 seconds. Perhaps we could minimize refresh to once per collection we're trying to balance. There are also several other places in the balancer code that we force refresh. They should all be audited and we should see if they can be minimized. The mongos and shard servers are more reactive to changes, invalidating and refreshing as needed. The config server is rather more spurious, in general. Maybe we should reconsider making it more reactive, only invalidating it when a chunk metadata change occurs, rather than forcing refreshes wherever freshness is a concern. |