[SERVER-4015] Chunk migration should be instantaneous when there's no data in the chunks Created: 04/Oct/11  Updated: 19/Nov/12  Resolved: 19/Nov/12

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Minor - P4
Reporter: Spencer Brody (Inactive) Assignee: Spencer Brody (Inactive)
Resolution: Duplicate Votes: 0
Labels: migration, sharding
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:

 Description   

If you pre-split a brand new collection into a large number of chunks, it will take a long time for the balancer to balance out those new chunks, because only one chunk is moved every few seconds. If there's no actual data to move, chunk migration is really just a metadata update to the config database, and should be able to be done instantaneously.



 Comments   
Comment by Eliot Horowitz (Inactive) [ 19/Nov/12 ]

SERVER-3602

Comment by Greg Studer [ 23/Feb/12 ]

Don't think we can do this instantaneously, at least in the non-initial split case. The key part of migration is to redirect any writes that could be going on as we perform the move - if we solely update the metadata without doing the locking on shards, new writes to the empty chunk could be misdirected.

Initial split actually has similar issues too - if we initially create chunks in different shards of the cluster, any writes coming in simultaneously for the unsharded collection won't correctly be redirected.

Comment by Spencer Brody (Inactive) [ 01/Nov/11 ]

Can just split the initial chunk into as many chunks as there are shards, assign one large chunk to each shard, then split those chunks up further as desired. This avoids needing to migrate a ton of empty chunks after doing a bunch of splits on a new collection.

Comment by Spencer Brody (Inactive) [ 04/Oct/11 ]

This would make pre-splitting before a large bulk import much simpler and faster to do.

Generated at Thu Feb 08 03:04:42 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.