[SERVER-66279] Use the BatchedDeleteStage in the range deleter Created: 06/May/22 Updated: 04/Jul/23 Resolved: 04/Jul/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Josef Ahmad | Assignee: | Pierlauro Sciarelli |
| Resolution: | Won't Do | Votes: | 0 |
| Labels: | range-deleter-improvements | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||
| Assigned Teams: |
Sharding EMEA
|
||||||||||||||||||||||||||||
| Sprint: | Sharding EMEA 2023-06-12, Sharding EMEA 2023-06-26, Sharding EMEA 2023-07-10 | ||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||
| Description |
|
The new BatchedDeleteStage deletes documents efficiently in batches. Dependencies that I'm aware of to switch the range deleter from the DeleteStage to the BatchedDeleteStage:
|
| Comments |
| Comment by Pierlauro Sciarelli [ 04/Jul/23 ] |
|
Closing as "won't do" because - after some internal testing - it was decided that the increase in latency caused by batched deletions (compared to doc-by-doc deletions) does not play well with the requirement that the range deleter must not impact user workload |
| Comment by Louis Williams [ 22/Dec/22 ] |
|
Within a range deleter batch, the BatchDeleteStage will further allow you to split up a multi-delete into its own batches. This can be tuned by specifying a doc limit and time limit. The batched deletes will probably not create significant performance improvements on workloads that are IO-bound, especially those deleting data that is out of cache. That said, batched deletes will reduce the journaling rate, so instead of one disk write request per document, we will only need one per batch. |
| Comment by Garaudy Etienne [ 22/Dec/22 ] |
|
whatever value we use for rangeDeleterBatchSize, will now be the number of documents we delete in a single batch. Therefore we should reduce the default batch size down to a reasonable value based on testing (anywhere between 128 and 5000?) since we will no longer be deleting documents one-at-a-time. |