[SERVER-33237] Optimize the Range Deleter speed Created: 09/Feb/18 Updated: 29/Oct/23 Resolved: 13/Jun/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | 3.2.19, 3.4.13, 3.6.3 |
| Fix Version/s: | 3.4.17, 3.6.7, 4.0.1, 4.1.1 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Kaloian Manassiev | Assignee: | Kevin Pulo |
| Resolution: | Fixed | Votes: | 2 |
| Labels: | bkp | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||
| Backwards Compatibility: | Minor Change | ||||||||||||||||||||
| Backport Requested: |
v4.0, v3.6, v3.4
|
||||||||||||||||||||
| Sprint: | Sharding 2018-04-09, Sharding 2018-05-07, Sharding 2018-05-21, Sharding 2018-06-04, Sharding 2018-06-18 | ||||||||||||||||||||
| Participants: | |||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||
| Linked BF Score: | 0 | ||||||||||||||||||||
| Description |
|
The sharding range deleter was intentionally implemented to not be fully efficient in order to minimize its interference with MMAP V1. This causes orphaned data to accumulate especially with the recent moveChunk improvements. With the WiredTiger storage engine, we can let it run at full speed and we should investigate the benefit of directly instantiating an auto-yielding delete stage instead of recreating the index scan phase for each document. |
| Comments |
| Comment by Kevin Pulo [ 28/Aug/18 ] |
|
The behaviour of this fix differs slightly between MongoDB 3.4 and 3.6+. In both cases, "batch" below refers to (by default) 128 document deletions performed by the Range Deleter.
Note that in 3.4, rangeDeleterBatchDelayMS occurs both when _secondaryThrottle is enabled and disabled (for consistency), even though the re-planning performance improvement only applies when _secondaryThrottle is disabled. This means that in 3.4, it may be necessary to adjust the value of rangeDeleterBatchDelayMS when changing between _secondaryThrottle enabled and disabled (and vice-versa), to avoid unexpected increases or decreases in Range Deleter performance. |
| Comment by Githook User [ 20/Aug/18 ] |
|
Author: {'name': 'Kevin Pulo', 'email': 'kevin.pulo@mongodb.com', 'username': 'devkev'}Message: Custom backport to v3.4, based on 6ed473ed3a122bebc1e932c946fe1c991dbd7ecb. In v3.4, the excessive query planning is only avoided when _secondaryThrottle Also add the rangeDeleterBatchDelayMS server parameter (default 20ms), to give |
| Comment by Githook User [ 11/Jul/18 ] |
|
Author: {'username': 'devkev', 'name': 'Kevin Pulo', 'email': 'kevin.pulo@mongodb.com'}Message: Rather than once for each deleted document. Also add the rangeDeleterBatchDelayMS server parameter (default 20ms), to give (cherry picked from commit 6ed473ed3a122bebc1e932c946fe1c991dbd7ecb) |
| Comment by Githook User [ 11/Jul/18 ] |
|
Author: {'username': 'devkev', 'name': 'Kevin Pulo', 'email': 'kevin.pulo@mongodb.com'}Message: Rather than once for each deleted document. Also add the rangeDeleterBatchDelayMS server parameter (default 20ms), to give (cherry picked from commit 6ed473ed3a122bebc1e932c946fe1c991dbd7ecb) |
| Comment by Githook User [ 13/Jun/18 ] |
|
Author: {'username': 'devkev', 'name': 'Kevin Pulo', 'email': 'kevin.pulo@mongodb.com'}Message: Rather than once for each deleted document. Also add the rangeDeleterBatchDelayMS server parameter (default 20ms), to give |