[SERVER-62368] Range deleter must honor rangeDeleterBatchDelayMS Created: 05/Jan/22  Updated: 29/Oct/23  Resolved: 31/Jan/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.3.0, 5.2.1, 4.4.13, 5.0.7

Type: Bug Priority: Major - P3
Reporter: Pierlauro Sciarelli Assignee: Pierlauro Sciarelli
Resolution: Fixed Votes: 0
Labels: sharding-wfbf-day
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Related
Backwards Compatibility: Fully Compatible
Backport Requested:
v5.2, v5.0, v4.4
Steps To Reproduce:

Unit test proving the erroneous behavior in v4.4 (diff applied on commit 3bdd66d).

diff --git a/src/mongo/db/s/range_deletion_util_test.cpp b/src/mongo/db/s/range_deletion_util_test.cpp
index e2296557eb2..8ccd2d7b17b 100644
--- a/src/mongo/db/s/range_deletion_util_test.cpp
+++ b/src/mongo/db/s/range_deletion_util_test.cpp
@@ -709,6 +709,11 @@ TEST_F(RangeDeleterTest, RemoveDocumentsInRangeRespectsDelayInBetweenBatches) {
         dbclient.insert(kNss.toString(), BSON(kShardKey << i));
     }
 
+    const ChunkRange range2(BSON(kShardKey << 10), BSON(kShardKey << 20));
+    for (auto i = 10; i < 10 + numDocsToInsert; i++) {
+        dbclient.insert(kNss.toString(), BSON(kShardKey << i));
+    }
+
     auto cleanupComplete =
         removeDocumentsInRange(executor(),
                                std::move(queriesComplete),
@@ -721,10 +726,25 @@ TEST_F(RangeDeleterTest, RemoveDocumentsInRangeRespectsDelayInBetweenBatches) {
                                Seconds(0) /* delayForActiveQueriesOnSecondariesToComplete */,
                                delayBetweenBatches);
 
+    auto cleanupComplete2 =
+        removeDocumentsInRange(executor(),
+                               std::move(queriesComplete),
+                               kNss,
+                               uuid(),
+                               kShardKeyPattern,
+                               range2,
+                               boost::none,
+                               numDocsToRemovePerBatch,
+                               Seconds(0) /* delayForActiveQueriesOnSecondariesToComplete */,
+                               Milliseconds(0) /* delayBetweenBatches */);
+
     // A best-effort check that cleanup has not completed without advancing the clock.
     sleepsecs(1);
     ASSERT_FALSE(cleanupComplete.isReady());
 
+    // The second task didn't wait for the delay scheduled by the first task
+    ASSERT_TRUE(cleanupComplete2.isReady());
+
     // Advance the time until cleanup is complete. This explicit advancement of the clock is
     // required in order to allow the delay between batches to complete. This cannot be made exact
     // because there's no way to tell when the sleep operation gets hit exactly, so instead we

Sprint: Sharding EMEA 2022-02-07
Participants:

 Description   

The semantic of the rangeDeleterBatchDelayMS parameter changed over time and the original idea of using it to throttle range deletions is not honored in the majority of scenarios.

Objective of this ticket is to introduce a global delay applied to the range deletions at thread pool level, independently from collections/ranges.

Following, a summary of the current semantics.

Versions gte v4.4

The delay is applied to the deletion of batches belonging to a specific range.

Problem: if there are several (more than one) range deletion tasks scheduled for different collections or for the same collection on different ranges, deletions are actually not throttled.

Versions until v4.2

The batch delay is per collection and the rescheduling of a cleanup task is not bound to a specific range.

Problem: if there are several (more than one) range deletion tasks scheduled for different collections, deletions are actually not throttled.



 Comments   
Comment by Githook User [ 03/Feb/22 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-62368 Range deleter must honor rangeDeleterBatchDelayMS
Branch: v4.4
https://github.com/mongodb/mongo/commit/d9144ddc222d1d8eabffca0094fa625f9d0a2a39

Comment by Githook User [ 03/Feb/22 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-62368 Range deleter must honor rangeDeleterBatchDelayMS
Branch: v5.2
https://github.com/mongodb/mongo/commit/b2ef16a8fc01c430af6dce82d4e75d97ada4d5fb

Comment by Githook User [ 03/Feb/22 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-62368 Range deleter must honor rangeDeleterBatchDelayMS
Branch: v5.0
https://github.com/mongodb/mongo/commit/d1eb6fb94ab4abd7a8d0954f24aaefe63392df05

Comment by Githook User [ 31/Jan/22 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-62368 Range deleter must honor rangeDeleterBatchDelayMS
Branch: master
https://github.com/mongodb/mongo/commit/1aad5e2e7c6e251de1d17d911c971fc1a1bb6bd0

Generated at Thu Feb 08 05:54:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.