Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-84769

Resharding remainingOpTime algorithm doesn't work with low elapsedTime

    • Cluster Scalability
    • Fully Compatible
    • ALL
    • v8.0, v7.0, v6.0, v5.0
    • Cluster Scalability 2024-5-13, Cluster Scalability 2024-5-27, Cluster Scalability 2024-6-10, Cluster Scalability 06/24/24, Cluster Scalability 2024-09-02, Cluster Scalability 2024-10-14, Cluster Scalability 2024-10-28, Cluster Scalability 2024-11-11

      In the following algorithm for calculating remainingTime to complete for resharding op, used by resharding commit monitor:

      Milliseconds remainingTime(Milliseconds elapsedTime, double elapsedWork, double totalWork) {
          elapsedWork = std::min(elapsedWork, totalWork);
          double remainingMsec = 1.0 * elapsedTime.count() * (totalWork / elapsedWork - 1);
          return Milliseconds(Milliseconds::rep(remainingMsec));
      }
      

      If the elapsedTime is of the order of few ms, the remainingMsec can be incorrectly reported. For example in the HELP-54235, with ~300k fetched oplog entries (totalWork) and a 1000 applied oplog entries (elapsedWork) and a value of elapsedTime as 6ms will result in engaging the CS as:

      remainingMsec = 1.0 * 6 * (300-1) ≈ 1800 ms = 1.8 seconds < 2 seconds.

      This algorithm needs to change to handle this edge case.

      Note: reshardingDelayBeforeRemainingOperationTimeQueryMillis parameter was introduced and backported to the releases attached in this ticket.
      In 8.0.4, it defaults to 30 seconds.
      In all other branches, it defaults to 0 seconds. Its default value will be changed to 30 seconds in the backports of SERVER-95311.

            Assignee:
            ben.gawel@mongodb.com Ben Gawel
            Reporter:
            abdul.qadeer@mongodb.com Abdul Qadeer
            Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved: