[SERVER-55028] Improve the auto-splitter policy Created: 08/Mar/21  Updated: 29/Oct/23  Resolved: 02/Sep/21

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 5.1.0-rc0

Type: Task Priority: Major - P3
Reporter: Pierlauro Sciarelli Assignee: Pierlauro Sciarelli
Resolution: Fixed Votes: 0
Labels: PM-2321-Chunk-Splitter
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-58650 Extract splitVector logic for the aut... Closed
depends on SERVER-58664 Simplify and comment autoSplitVector Closed
Duplicate
is duplicated by SERVER-52886 Sharded cluster is balancing a lot an... Closed
Related
is related to SERVER-60009 Increase auto splitted chunks size Closed
Backwards Compatibility: Fully Compatible
Sprint: Sharding EMEA 2021-07-26, Sharding EMEA 2021-08-09, Sharding EMEA 2021-08-23, Sharding EMEA 2021-09-06
Participants:
Case:

 Description   

The chunk splitter is currently relying on the splitVector function that can easily suggest to always split at a chunk at (maxChunkSize / 2).

While the documentation states that a chunk gets partitioned when it reaches the maximum chunk size, the current implementation can force a split simply if the current size is (maxChunkSize / 2 + ε). This results in the max chunk size being actually (maxChunkSize / 2).

Not only that, some corner cases can produce very large chunk counts relative to document count.

This ticket has two objectives to make the auto-splitter less aggressive:

  • As a precondition for splitting, wait for a chunk size to get closer to the maximum
  • Consider making bigger chunks, with a size closer to the maxChunkSize set by the user rather than half of it


 Comments   
Comment by Vivian Ge (Inactive) [ 06/Oct/21 ]

Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you!

Comment by Githook User [ 02/Sep/21 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-55028 Improve the auto-splitter policy
Branch: master
https://github.com/mongodb/mongo/commit/e97122db1397fbf7f6b520de27445ba3e5a5b156

Comment by Kaloian Manassiev [ 01/Apr/21 ]

pierlauro.sciarelli to rewrite the description.

Generated at Thu Feb 08 05:35:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.