[SERVER-61557] Get rid of top chunk migration Created: 17/Nov/21  Updated: 16/May/23  Resolved: 20/Mar/23

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Pierlauro Sciarelli Assignee: Pierlauro Sciarelli
Resolution: Duplicate Votes: 0
Labels: shardingemea-qw
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-66652 Remove NoMoreAutoSplitter feature flag Closed
is depended on by SERVER-65322 Get rid of legacy `moveChunk` path on... Closed
Duplicate
duplicates SERVER-66652 Remove NoMoreAutoSplitter feature flag Closed
is duplicated by SERVER-44193 Remove Top Chunk Optimisation Closed
Related
related to SERVER-75065 Complete TODO listed in SERVER-61557 Closed
related to SERVER-44192 Introduce metrics to measure the usef... Closed
Assigned Teams:
Sharding EMEA
Sprint: Sharding EMEA 2023-02-20, Sharding EMEA 2023-03-06, Sharding EMEA 2023-03-20
Participants:
Story Points: 3

 Description   

The top chunk migration was thought as a way to distribute the write load for applications inserting a lot of documents with non-hashed incremental shard keys.

It turns out that for heavy workloads it could cause more harm than good:

  • Trying to move the right-most chunk while writing on it could result in very long migrations that can't reach the catch-up phase and eventually fail after hours.
  • In a balanced environment, it results in ping-pong (moving continuously the top chunk on another shard, then split, then move back) that can bring to increased migrations unavailability due to the waiting for overlapping range deletions to finish on the receiver.


 Comments   
Comment by Githook User [ 16/May/23 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-75065 Complete TODO listed in SERVER-61557
Branch: master
https://github.com/mongodb/mongo/commit/31572586adb8aac04d930ca56887ca742bd2c048

Comment by Pierlauro Sciarelli [ 20/Mar/23 ]

Closing as duplicate of SERVER-66652 that got rid of the chunk splitter that was the only component issuing top chunk migrations

Comment by Pierlauro Sciarelli [ 31/May/22 ]

I would keep separate tickets.

We protect only the triggering of the top chunk migration flow via the feature flag, but not all code used for performing top chunk optimization is protected by the feature flag. For example, Balancer::rebalanceSingleChunk is exclusively used by top chunk optimization, but there is no reference to the feature flag.

Comment by Tommaso Tocci [ 31/May/22 ]

pierlauro.sciarelli@mongodb.com could you confirm that the entire code of the top chunk optimization will go away with the autosplitter? If so can we mark this as duplicate of the NOMAS feature flag removal

Comment by Kaloian Manassiev [ 18/Nov/21 ]

I would like to turn this into a project around improving the behaviour of bulk-loading of data into a new sharded cluster.

Generated at Thu Feb 08 05:52:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.