[SERVER-84769] Resharding remainingOpTime algorithm doesn't work with low elapsedTime Created: 11/Jan/24  Updated: 31/Jan/24

Status: Needs Scheduling
Project: Core Server
Component/s: Sharding
Affects Version/s: 5.0.0, 6.0.0, 7.0.0, 7.2.0
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Abdul Qadeer Assignee: Backlog - Cluster Scalability
Resolution: Unresolved Votes: 0
Labels: cs-80-targeted, cs-subteam1
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Assigned Teams:
Cluster Scalability
Operating System: ALL
Participants:

 Description   

In the following algorithm for calculating remainingTime to complete for resharding op, used by resharding commit monitor:

Milliseconds remainingTime(Milliseconds elapsedTime, double elapsedWork, double totalWork) {
    elapsedWork = std::min(elapsedWork, totalWork);
    double remainingMsec = 1.0 * elapsedTime.count() * (totalWork / elapsedWork - 1);
    return Milliseconds(Milliseconds::rep(remainingMsec));
}

If the elapsedTime is of the order of few ms, the remainingMsec can be incorrectly reported. For example in the HELP-54235, with ~300k fetched oplog entries (totalWork) and a 1000 applied oplog entries (elapsedWork) and a value of elapsedTime as 6ms will result in engaging the CS as:

remainingMsec = 1.0 * 6 * (300-1) ≈ 1800 ms = 1.8 seconds < 2 seconds.

This algorithm needs to change to handle this edge case.


Generated at Thu Feb 08 06:55:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.