[SERVER-8335] remember _dataWritten for unaffected chunks after partial chunk manager reload Created: 25/Jan/13  Updated: 11/Jul/16  Resolved: 18/Feb/13

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 2.2.4, 2.4.0-rc1

Type: Question Priority: Major - P3
Reporter: Greg Studer Assignee: Greg Studer
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File hash_autosplit.js    
Issue Links:
Related
is related to SERVER-8187 Usability issues with splitting + has... Closed
Participants:

 Description   

for QA, ensure autosplit behavior is as-designed, in particular when initially loading a collection with hashed shard keys.



 Comments   
Comment by auto [ 21/Mar/13 ]

Author:

{u'date': u'2013-02-18T17:55:42Z', u'name': u'Greg Studer', u'email': u'greg@10gen.com'}

Message: SERVER-8335 test for autosplit heuristics
Branch: v2.2
https://github.com/mongodb/mongo/commit/ab40103bbb365075ab5300692ad6412c3c038040

Comment by auto [ 21/Mar/13 ]

Author:

{u'date': u'2013-02-11T22:27:33Z', u'name': u'Greg Studer', u'email': u'greg@10gen.com'}

Message: SERVER-8335 remember _dataWritten for unaffected chunks after partial chunk manager reload
Branch: v2.2
https://github.com/mongodb/mongo/commit/aa3eab2d92a96d13890fa84b524cd48990ce24c0

Comment by auto [ 18/Feb/13 ]

Author:

{u'date': u'2013-02-18T17:55:42Z', u'name': u'Greg Studer', u'email': u'greg@10gen.com'}

Message: SERVER-8335 test for autosplit heuristics
Branch: master
https://github.com/mongodb/mongo/commit/73805ae6a4bf4b953d79ffbca70bb427b114491b

Comment by auto [ 14/Feb/13 ]

Author:

{u'date': u'2013-02-11T22:27:33Z', u'name': u'Greg Studer', u'email': u'greg@10gen.com'}

Message: SERVER-8335 remember _dataWritten for unaffected chunks after partial chunk manager reload
Branch: master
https://github.com/mongodb/mongo/commit/ea4b79b64fc4b7fd80aa5b4ec171ac8a37190801

Comment by Greg Studer [ 11/Feb/13 ]

Think this is unrelated to hashed sharding in particular, but related to aggressively resetting the heuristic data we use for splitting. Test case outputs "chunk too big" errors using original build, no errors with patched build.

Comment by Kristina Chodorow (Inactive) [ 28/Jan/13 ]

I was using a 1MB chunk size, it split a bunch on each shard (just not enough).

Comment by Kevin Matulef [ 28/Jan/13 ]

You probably need to insert more than 1 million docs for mongos auto-splitting to kick in (correct me if I'm wrong).

Comment by Kristina Chodorow (Inactive) [ 25/Jan/13 ]

Yes: made 5 shards, create a hashed shard key on user_id, populated with 1 million docs of the form

{user_id:"user"+Math.floor(Math.random()*10000), date:new Date(), something:Math.random()}.

When the inserts finished, every chunk I tried to move was too big.

Generated at Thu Feb 08 03:17:09 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.