[SERVER-47699] Change yield type used by range deleter from YIELD_MANUAL to YIELD_AUTO Created: 22/Apr/20 Updated: 29/Oct/23 Resolved: 27/Apr/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | None |
| Fix Version/s: | 4.0.25, 4.2.15, 4.4.7, 5.0.0-rc0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Eric Sommer | Assignee: | Antonio Fuschetto |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | sharding-wfbf-day | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Backport Requested: |
v4.4, v4.2, v4.0
|
||||||||||||||||
| Sprint: | Sharding 2021-04-05, Sharding EMEA 2021-05-03 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||
| Linked BF Score: | 154 | ||||||||||||||||
| Description |
|
Currently, when deleting batches of documents (default 128) the range deleter only yields manually. This is different behavior than regular bulk deletes, which yield on time as well. The range deleter should also yield on time, making it functionally equivalent to regular bulk deletes. Failing to do so could result in locks being held and blocking other operations longer than anticipated in a case where range deletion was unexpectedly slow (due to an i/o bottleneck, for example). |
| Comments |
| Comment by Githook User [ 11/May/21 ] |
|
Author: {'name': 'Antonio Fuschetto', 'email': 'antonio.fuschetto@mongodb.com', 'username': 'afuschetto'}Message: |
| Comment by Githook User [ 05/May/21 ] |
|
Author: {'name': 'Antonio Fuschetto', 'email': 'antonio.fuschetto@mongodb.com', 'username': 'afuschetto'}Message: |
| Comment by Githook User [ 05/May/21 ] |
|
Author: {'name': 'Antonio Fuschetto', 'email': 'antonio.fuschetto@mongodb.com', 'username': 'afuschetto'}Message: |
| Comment by Kaloian Manassiev [ 29/Apr/21 ] |
|
Approving backport down to versions 4.4 and 4.2 until we evaluate whether the same correctness reasoning applies for 4.0 as well (the range deleter changed in 4.2 and 4.4). |
| Comment by Githook User [ 27/Apr/21 ] |
|
Author: {'name': 'Antonio Fuschetto', 'email': 'antonio.fuschetto@mongodb.com', 'username': 'afuschetto'}Message: |
| Comment by Andy Schwerin [ 26/Apr/21 ] |
|
Requesting consideration for backport to 4.4, 4.2 and 4.0, in case kaloian.manassiev was right that it should be safe to all those branches. |
| Comment by Githook User [ 23/Apr/21 ] |
|
Author: {'name': 'Antonio Fuschetto', 'email': 'antonio.fuschetto@mongodb.com', 'username': 'afuschetto'}Message: |
| Comment by Kaloian Manassiev [ 23/Mar/21 ] |
|
After discussion with schwerin, I believe that this work can actually be simpler that I thought, so I am bringing the ticket up to 'Needs Triage' on the EMEA backlog to be done on WFBF day. Tl;dr: Because of this it is not possible for a chunk to be moved inside the range during yield. So in other words, we can actually just enable auto yielding in 4.4 and master and it will just work. For 4.2 and 4.0, the range deleter does similar check against the in-memory metadata and so does chunk receive, so we should be able to backport that change there as well. For both releases, the only thing that's a bit unclear to me is what happens if a drop happens during the yield. I imagine that the plan executor will get killed and that would cause the loop to retry and discover the drop, but whoever picks-up this work should confirm that. |
| Comment by Andrew Shuvalov (Inactive) [ 25/Feb/21 ] |
|
I wonder if As was explained in comment in the version 4.2 the range deletion was naively prorated by blocking operations. Making it more asynchronous in 4.4 created a problem. Doing this in async way is still correct, but the rate limiter should prevent the next batch, especially a batch from unrelated range delete, to start too early. |