[SERVER-61637] Review range deleter batching policy Created: 19/Nov/21  Updated: 29/Oct/23  Resolved: 14/Dec/21

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 5.3.0, 5.1.2, 5.0.6, 5.2.0-rc3, 4.4.12

Type: Task Priority: Major - P3
Reporter: Pierlauro Sciarelli Assignee: Pierlauro Sciarelli
Resolution: Fixed Votes: 0
Labels: range-deleter
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Documented
is documented by DOCS-15059 Update rangeDeleterBatchSize default ... Closed
Problem/Incident
causes SERVER-62091 Set small range deleter batch size in... Closed
Related
Backwards Compatibility: Fully Compatible
Backport Requested:
v5.2, v5.1, v5.0, v4.4, v4.2, v4.0
Sprint: Sharding EMEA 2021-12-13, Sharding EMEA 2021-12-27
Participants:
Linked BF Score: 135

 Description   

With the changes committed under SERVER-47699 (auto-yielding), it should be safe to get rid of batched deletes and - instead - simply delete documents belonging to a whole range in one shot. Some experiments showed that deleting hundred of thousands of documents takes less than a minute and the yielding should theoretically prevent any kind of starvation.

Keeping the current behavior of the range-deleter can easily result in the balancer starving waiting for a specific range deletion to drain (the more range deletions, the more likely this happens - because the task gets re-enqueued behind all the other ones).

Objective of this ticket is to decide whether we need to:

- Get rid of the batching policy
- Keep it, but drastically increasing rangeDeleterBatchSize (e.g. set it to 100K). This may be useful in case
a user wants to throttle orphan deletions in favor of CRUD operations.



 Comments   
Comment by Githook User [ 24/Dec/21 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-61637 Increase default range deletion batch size
Branch: v4.4
https://github.com/mongodb/mongo/commit/a17699e50a4d75d91b5788baa7eb15916341b905

Comment by Githook User [ 24/Dec/21 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-61637 Increase default range deletion batch size
Branch: v5.1
https://github.com/mongodb/mongo/commit/073d0a252ed3e820e685fe227779400881df17bf

Comment by Githook User [ 24/Dec/21 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-61637 Increase default range deletion batch size
Branch: v5.0
https://github.com/mongodb/mongo/commit/725a7f5045f4c9f48a4490825df296765207e096

Comment by Githook User [ 24/Dec/21 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-61637 Increase default range deletion batch size
Branch: v5.2
https://github.com/mongodb/mongo/commit/87f9db51df95d631cecdb89e68de181cdd39044b

Comment by Pierlauro Sciarelli [ 14/Dec/21 ]

Comment for posterity: it has been decided to set the batch size to unlimited (max int) but still leaving the possibility to tune the rangeDeleterBatchSize parameter in case someone wants to throttle range deletions.

Comment by Githook User [ 14/Dec/21 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-61637 Increase default range deletion batch size
Branch: master
https://github.com/mongodb/mongo/commit/edc61886e4c37edccd77afaa3e3ed255dc93a61c

Generated at Thu Feb 08 05:52:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.