Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-2647

Balancing strategy based on available shards rather than chunk size

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Sharding
    • Labels:
      None

      Description

      Currently we need sharding solely for write distribution (due to high write lock contention). Due to the fact that these writes happen on relatively small collections automatic sharding fails because collections may never top even 10mb in size so chunks are never split, and therefore never moved across shards. I'm aware we could use even smaller chunk sizes to force the issue but I was led to understood frequent chunk migrations come with significant overhead.

      I was wondering if it's possible to add a balancing strategy that would reserve a chunk on each available shard for a collection and start distributing writes. Chunk moves/splits should only occur if additional shards are added and/or the chunk size exceeds a specific size (the current strategy, basically).

      I'm not sure if I'm explaining this right but basically what I want is :
      1) Distributes writes from the very first update/insert
      2) Rebalanced when shards are added or removed

      Hope it makes sense. If there's a solution available to reach the same goal that doesn't involve manual admin I'd be interested in hearing it. Thanks

        Attachments

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            remonvv Remon van Vliet
            Participants:
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: