[SERVER-5507] Add sharded per-collection settings (chunksize/balancing/window) Created: 05/Apr/12  Updated: 16/Jan/24

Status: Blocked
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Scott Hernandez (Inactive) Assignee: Backlog - Catalog and Routing
Resolution: Unresolved Votes: 1
Labels: balancer, chunks, sharding
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
is duplicated by SERVER-45630 Allow specifying a chunkSize setting ... Closed
is duplicated by SERVER-5277 chunkSize per sharded collection option Closed
Related
is related to SERVER-51890 Support multiple balancing windows wi... Closed
is related to SERVER-56464 Allow to disable/enable autosplit per... Closed
Assigned Teams:
Catalog and Routing
Participants:
Case:

 Description   

Not every collection needs to be balanced in the same ways. The maxChunkSize for example will depend on the avg doc size in many cases.



 Comments   
Comment by Tommaso Tocci [ 04/Nov/21 ]

SERVER-58270 introduced a new command configureCollectionAutoSplitter that allows to set per collection max chunk size.

Comment by Esha Maharishi (Inactive) [ 02/Mar/20 ]

Current state of the world

Global settings

The following auto-balancing settings are global (not per namespace), and are stored in three documents in the config.settings collection on the config server:

  • A document with _id: "balancer", which is described by BalancerSettingsType, which includes the balancing mode (full, autosplit-only, off), active window, and whether to balance jumbo chunks

Every cluster node has an in-memory cache of these settings in an instance of BalancerConfiguration, which lives on the Grid.

Mongos periodically refreshes its BalancerConfiguration under the ShardingUptimeReporter thread, which is launched on mongos startup.

I actually am not sure why mongos needs to know the BalancerConfiguration - kaloian.manassiev, renctan, is it unnecessary now that neither autosplitting nor autobalancing are done by mongos?

Shard primaries periodically refresh their BalancerConfiguration under the PeriodicBalancerConfigRefresher thread, which is launched on sharding initialization, paused on stepdown, and resumed on stepup.

On shards, the ShardServerOpObserver calls incrementChunkOnInsertOrUpdate, which checks whether to initiate a split attempt.

Config servers  periodically refresh their BalancerConfiguration under the balancer thread.

On config servers, the balancer thread checks whether to do a balancing round and whether to balance jumbo chunks.

Per-collection settings

Additionally, there is a per-collection setting for whether to allow auto-balancing, which is stored in each entry in config.collections.

Shards check this setting to determine whether to opt out of auto-splitting for a specific collection.

Config servers check this setting to determine whether to opt out of auto-balancing for a specific collection.

Generated at Thu Feb 08 03:09:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.