[SERVER-53736] Resumable range deleter must use timestamps Created: 13/Jan/21  Updated: 06/Dec/22  Resolved: 14/Jan/21

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 4.9.0, 4.4.3
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Pierlauro Sciarelli Assignee: [DO NOT USE] Backlog - Sharding Team
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Sharding
Operating System: ALL
Participants:

 Description   

By default, each range deletion task is scheduled on the executor at time (timestampMoveChunkFinished + 15 mins). If a stepdown happens, at stepUp all pending range deletions are re-submitted at time (timeOfResubmit + 15 mins) because the only persisted info is if a range deletion is delayed but not when it should have originally happened.

This behavior makes it very easy to end up into problematic contexts such as:

  • If a sufficient number of range deletions are rescheduled, the collection could be under pressure (because continuously IX-locked) and the secondaries get very lagged because there is no syncrhonous wait for majority deletion and there are no timeouts between batches from different ranges.
  • If periodic stepdowns are happening (e.g. a stepdown every 10 mins), range deletions will never be performed.


 Comments   
Comment by Ratika Gandhi [ 14/Jan/21 ]

We will take care of this as a part of the larger undertaking on balancers. 

Generated at Thu Feb 08 05:31:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.