[SERVER-9365] mongod always split at 250000 position Created: 17/Apr/13  Updated: 11/Jul/16  Resolved: 23/Jul/13

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 2.2.1
Fix Version/s: 2.4.6, 2.5.2

Type: Bug Priority: Major - P3
Reporter: peanutgyz Assignee: Greg Studer
Resolution: Done Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
is duplicated by SERVER-9498 Possible bug in SplitVector: Mongodb ... Closed
is duplicated by SERVER-9690 SplitVector fails to find the mid-poi... Closed
is duplicated by SERVER-9792 Wrong maxChunkSize on SplitVector w/ ... Closed
Related
related to SERVER-10271 jstests/sharding/count1.js failing on... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:

 Description   

use mongod store small values , average size is 80 bytes,
chunk size is 64MB.

when add new shard into cluster, moveChunk start, and always say chunk is too large,

can't move, then split the chunk with force:true option.

mongod log:
limiting split vector to 250000 (from 634775618) objects

634775618 seems to large! it is half of the collection objects count.

i found in source code ,

s/d_split.cpp

const long long recCount = d->stats.nrecords;
const long long dataSize = d->stats.datasize;
 // 'force'-ing a split is equivalent to having maxChunkSize be the size of the current chunk, i.e., the
 // logic below will split that chunk in half
 long long maxChunkSize = 0;
 bool force = false;
 {
     BSONElement maxSizeElem = jsobj[ "maxChunkSize" ];
     BSONElement forceElem = jsobj[ "force" ];
 
     if ( forceElem.trueValue() ) {
         force = true;
         maxChunkSize = dataSize;
 
     }

when force is true, it will set maxChunkSize to the chunk size.

but i think dataSize is the size of whole collection , not the chunk,

so when force is true, it will always split at 250000 pos, not at 1/2 of the chunk.

am i wrong ?



 Comments   
Comment by auto [ 02/Aug/13 ]

Author:

{u'username': u'gregstuder', u'name': u'Greg Studer', u'email': u'greg@10gen.com'}

Message: SERVER-9365 make forced halfway split work with chunks with many docs

Additional test fixes.
Branch: v2.4
https://github.com/mongodb/mongo/commit/9a7944042ca58ed776a501cd02910bc384ce4a6c

Comment by auto [ 22/Jul/13 ]

Author:

{u'username': u'gregstuder', u'name': u'Greg Studer', u'email': u'greg@10gen.com'}

Message: SERVER-9365 make forced halfway split work with chunks with many docs

Additional test fixes.
Branch: master
https://github.com/mongodb/mongo/commit/b51ad40a7b582baea1ba25b46020f6b4275878e9

Comment by auto [ 20/Jul/13 ]

Author:

{u'username': u'gregstuder', u'name': u'Greg Studer', u'email': u'greg@10gen.com'}

Message: Revert "SERVER-9365 make forced halfway split work with chunks with many docs"

This reverts commit 77ab6a325498a7dd8f4a60e4c7cf1c6385f1ad98.
Branch: master
https://github.com/mongodb/mongo/commit/9a4ac2f650a3c55218a50dd275665f81b016e7d9

Comment by Johan Hedin [ 19/Jul/13 ]

Thanks for the feedback Greg!

I'm running a patch myself that's identical to your fix so I'm temporary fine. Good to see a proper solution though.

Comment by Greg Studer [ 19/Jul/13 ]

@johan - fix is slated for backport, not yet clear when it will get to 2.4

Comment by auto [ 19/Jul/13 ]

Author:

{u'username': u'gregstuder', u'name': u'Greg Studer', u'email': u'greg@10gen.com'}

Message: SERVER-9365 make forced halfway split work with chunks with many docs
Branch: master
https://github.com/mongodb/mongo/commit/77ab6a325498a7dd8f4a60e4c7cf1c6385f1ad98

Comment by Johan Hedin [ 18/Jun/13 ]

Will the fix for this be in 2.4.5?

Comment by Johan Hedin [ 29/Apr/13 ]

This seem to be related as well: https://groups.google.com/d/topic/mongodb-user/nJG8WlCdta0/discussion

Comment by Johan Hedin [ 29/Apr/13 ]

This seem to bee the same problem as https://jira.mongodb.org/browse/SERVER-9498

Generated at Thu Feb 08 03:20:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.