[SERVER-2647] Balancing strategy based on available shards rather than chunk size Created: 01/Mar/11  Updated: 02/Mar/11  Resolved: 02/Mar/11

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Remon van Vliet Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:

 Description   

Currently we need sharding solely for write distribution (due to high write lock contention). Due to the fact that these writes happen on relatively small collections automatic sharding fails because collections may never top even 10mb in size so chunks are never split, and therefore never moved across shards. I'm aware we could use even smaller chunk sizes to force the issue but I was led to understood frequent chunk migrations come with significant overhead.

I was wondering if it's possible to add a balancing strategy that would reserve a chunk on each available shard for a collection and start distributing writes. Chunk moves/splits should only occur if additional shards are added and/or the chunk size exceeds a specific size (the current strategy, basically).

I'm not sure if I'm explaining this right but basically what I want is :
1) Distributes writes from the very first update/insert
2) Rebalanced when shards are added or removed

Hope it makes sense. If there's a solution available to reach the same goal that doesn't involve manual admin I'd be interested in hearing it. Thanks



 Comments   
Comment by Remon van Vliet [ 02/Mar/11 ]

Great, will add my vote to that. Would be good to add a comment to that issue elaborating on it a bit. Avoids duplicates

Comment by Eliot Horowitz (Inactive) [ 02/Mar/11 ]

SERVER-2001 will result in what you are looking for

Generated at Thu Feb 08 03:00:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.