[SERVER-7358] Pre-split new collections when using a hashed shard key Created: 15/Oct/12  Updated: 05/Jun/13  Resolved: 20/Nov/12

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 2.3.0
Fix Version/s: 2.3.1

Type: Improvement Priority: Major - P3
Reporter: Kevin Matulef Assignee: Kevin Matulef
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-2001 option to hash shard key Closed
Related
related to DOCS-1398 hashed shard key, add link/blurb abou... Closed
related to SERVER-3602 Add fast-path for migrating empty chunks Closed
is related to DOCS-762 Document hashed shard key Closed
Backwards Compatibility: Fully Compatible
Participants:

 Description   

When using a hashed shard key on a new collection, we can pre-split the range into many chunks, and redistribute those chunks before data is inserted, to avoid shuffling data around later. Users can currently do this manually by calling the 'split' command before inserting data, but we should do it automatically on new collections.



 Comments   
Comment by auto [ 20/Nov/12 ]

Author:

{u'date': u'2012-11-20T21:47:43Z', u'email': u'matulef@gmail.com', u'name': u'Kevin Matulef'}

Message: SERVER-7358 ammend comments to make warnings more accurate
Branch: master
https://github.com/mongodb/mongo/commit/518ac5389697045ca7262b19726a4905814a8d18

Comment by auto [ 20/Nov/12 ]

Author:

{u'date': u'2012-11-20T20:49:38Z', u'email': u'matulef@10gen.com', u'name': u'Kevin Matulef'}

Message: SERVER-7358 fix issue with mrShardedOutput.js and pre-splitting

An earlier commit ensured that shardcollection only put initial
chunks on the primary shard by default. This is safest, since
there may be concurrent inserts into the collection, but it's not
the behavior that map-reduce expects. This commit allows the caller
of shardcollection to specify a set of initial shards to place
chunks on. It's safe to use here because the inserts generated by
map-reduce all come from the same source.
Branch: master
https://github.com/mongodb/mongo/commit/d20100997e519ad744cfbef48cc1b5010d98f94e

Comment by auto [ 20/Nov/12 ]

Author:

{u'date': u'2012-11-20T19:46:03Z', u'email': u'greg@10gen.com', u'name': u'Greg Studer'}

Message: SERVER-7358 buildbot zBigMapReduce.js don't presplit automatically unless shard key is hashed
Branch: master
https://github.com/mongodb/mongo/commit/b73765f0fe003ca92508552283dbbd8a6e7ff87b

Comment by auto [ 20/Nov/12 ]

Author:

{u'date': u'2012-11-20T03:41:48Z', u'email': u'matulef@gmail.com', u'name': u'Kevin Matulef'}

Message: SERVER-7358 handle edge case where chunk to move is not on primary
Branch: master
https://github.com/mongodb/mongo/commit/273d8c921532b67ba9291c470732d2d56ee116ee

Comment by auto [ 20/Nov/12 ]

Author:

{u'date': u'2012-11-20T00:37:26Z', u'email': u'matulef@gmail.com', u'name': u'Kevin Matulef'}

Message: SERVER-7358 pre-split new colls when using a hashed shard key
Branch: master
https://github.com/mongodb/mongo/commit/3db6cf6bcccf838044d6c7f6460fd4e90ba9ad1d

Comment by auto [ 16/Nov/12 ]

Author:

{u'date': u'2012-11-16T00:31:27Z', u'name': u'Kevin Matulef', u'email': u'matulef@gmail.com'}

Message: SERVER-7358 SERVER-3602 make empty chunk movements faster
Branch: master
https://github.com/mongodb/mongo/commit/e8cee07ef063b849e89b507dbcb618d48b7cd2fe

Generated at Thu Feb 08 03:14:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.