Details
-
Improvement
-
Resolution: Duplicate
-
Minor - P4
-
None
-
None
Description
Sharding doesn't make sense data with this kind of model.
User had interesting case of sharded cluster, two shards, single database with huge number of small collections. Shard key was random, non-increasing. "Primary shard" for this database was getting overloaded, balancer wasn't moving chunks off. Cause is obvious: each collection only had one or two chunks, so balancer never considered the collection imbalanced. The cluster as a whole was tragically imbalanced.
Solution: pre-split and move chunks for half the collections. Or create a second database, movePrimary it to the other shard, and create half the new collections on this second database so their data starts on the second shard as their default home.
Better solution: don't create thousands of small collections on a sharded cluster, ensure each collection is large enough to get balanced – more than 20-30 chunks at least.
Attachments
Issue Links
- is duplicated by
-
DOCS-761 Best Practices on Sharding Large Collections
-
- Closed
-