[SERVER-59931] Improve defragmentation script Created: 14/Sep/21  Updated: 29/Oct/23  Resolved: 01/Oct/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.1.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Simon Gratzer (Inactive) Assignee: Simon Gratzer (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Participants:

 Description   

Defragmentation script

Python degragmentation script improvements:

  • Phase 1 is not idempotent thus executing the same phase twice will over merge chunks, possibly producing chunks with sizes that are double the target chunk size.
  • Phase 2 always splits chunks in half without taking into account the real chunk size.
  • There is no way of throttling the migrations performed in phase 2
  • The progression bar is not accurate because it always accounts for all the chunks in the collection even if they are not going to be migrated.

Performance evaluation
In order to evaluate the impact of migrations after the defragmentation process we decided to run a few migrations after phase 1 and measure the time taken by the catalog cache refreshes and possibly the impact on a synthetic read workload. In order to perform this evaluation we will provide a new version of the defragmentation script with the following improvements:

  • Allow to run each phase separately, so that it will be possible to assess the performance after each phase.
  • Expose a parameter to limit the number of migrations performed in phase 2. So that the script can be also used to easily trigger a specific amount of migrations.


 Comments   
Comment by Vivian Ge (Inactive) [ 06/Oct/21 ]

Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you!

Generated at Thu Feb 08 05:48:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.