[SERVER-30572] Support configurable 'jumbo' chunk threshold Created: 09/Aug/17  Updated: 29/Jan/18  Resolved: 08/Nov/17

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Kaloian Manassiev Assignee: Randolph Tan
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
Related
is related to SERVER-19919 Chunks that exceed 250000 docs but ar... Closed
Backwards Compatibility: Fully Compatible
Participants:

 Description   

Currently if a chunk spans over more than 250,000 documents it will be considered 'jumbo' and the balancer will refuse to move it until it is split into smaller chunks. In the case of bad shard key selection where the key cardinality is too high, jumbo chunks may arise which cannot be split because the entire chunk consists of the same shard key value.

The hardcoded 250,000 documents limit was included as a precaution against such large chunks taking too long to migrate and causing the donor shard to run out of memory because of accumulation of 'mods'.

In order to allow users in emergency situations to move jumbo chunks out of shards we should allow this limit to be temporarily increased. If we do this, we should also put a cap on the size of accumulated in-memory mods.



 Comments   
Comment by Randolph Tan [ 08/Sep/17 ]

Note: this will no longer be needed if we get rid of the limit completely in SERVER-19919.

Generated at Thu Feb 08 04:24:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.