-
Type:
Improvement
-
Resolution: Won't Fix
-
Priority:
Major - P3
-
None
-
Affects Version/s: 2.2.3
-
Component/s: Sharding
-
None
-
None
-
3
-
None
-
None
-
None
-
None
-
None
-
None
As an adjunct to the ability to pre-split a collection, it would be helpful to be able to specify the migrations that will distribute the chunks to the shards on which they will reside.
The moveChunk command will move a single chunk, but a second moveChunk command will fail if issued before the first migration has finished. Moreover, if the chunk migration fails for any reason, it is not remembered by the cluster and so migrations must be monitored and failed migrations re-requested.
This is particularly noticeable when used with pre-splitting, in which the "chunk migrations" are of empty chunks and therefore do not require data movement, merely changes to the chunks collection.
It would be better to be able to specify a queue of desired migrations, and then let MongoDB do the work to make sure that the migrations happen as requested.
An additional complication for this use case is that these queue requests may come from any client at any time, so the single cluster-wide queue should accept multiple requests and add them to the queue without regard for whether there are already queued requests.
From the original request:
We are struggling with operations to provision and destroy (then re-provision) databases because of the current limitation in Mongo of moving one chunk at a time. We'd like the ability to have move chunk operations (especially empty chunks, which is what are trying to do when provisioning a database) to be queued so that the move operation eventually completes. Right now, the fact that a second move fails (and fail silently) is causing us to consider adding a queuing mechanism to our application code for something that seems like it should be supported in the database.