Uploaded image for project: 'Documentation'
  1. Documentation
  2. DOCS-585

FAQ: Sharding with Large Number of Small Collections

    XMLWordPrintableJSON

Details

    • Icon: Improvement Improvement
    • Resolution: Duplicate
    • Icon: Minor - P4 Minor - P4
    • Server_Docs_20231030
    • None
    • manual
    • None

    Description

      Sharding doesn't make sense data with this kind of model.

      User had interesting case of sharded cluster, two shards, single database with huge number of small collections. Shard key was random, non-increasing. "Primary shard" for this database was getting overloaded, balancer wasn't moving chunks off. Cause is obvious: each collection only had one or two chunks, so balancer never considered the collection imbalanced. The cluster as a whole was tragically imbalanced.

      Solution: pre-split and move chunks for half the collections. Or create a second database, movePrimary it to the other shard, and create half the new collections on this second database so their data starts on the second shard as their default home.

      Better solution: don't create thousands of small collections on a sharded cluster, ensure each collection is large enough to get balanced – more than 20-30 chunks at least.

      Attachments

        Activity

          People

            Unassigned Unassigned
            jesse@mongodb.com A. Jesse Jiryu Davis
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:
              7 years, 39 weeks, 1 day ago