[SERVER-37720] Allow active index builds during renameCollection Created: 24/Oct/18 Updated: 04/Dec/23 |
|
| Status: | Blocked |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Dianna Hohensee (Inactive) | Assignee: | Backlog - Storage Execution Team |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Assigned Teams: |
Storage Execution
|
||||||||||||||||||||||||
| Sprint: | Execution Team 2019-06-17, Execution Team 2019-07-01, Execution Team 2019-07-15, Execution Team 2019-07-29, Execution Team 2019-08-12, Execution Team 2019-12-02, Execution Team 2019-12-16, Execution Team 2020-01-13, Execution Team 2020-01-27, Execution Team 2020-02-10, Execution Team 2019-12-30 | ||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Linked BF Score: | 19 | ||||||||||||||||||||||||
| Description |
|
We expect allowing index builds during rename collection to be fairly straightforward now that This task will be a little exploratory as no has taken a look at the code. |
| Comments |
| Comment by Gregory Wlodarek [ 27/May/20 ] | ||||||||
|
We aren’t able to do this ticket just yet as queries do not survive across collection renames. Once SERVER-31695 is finished, we can revisit this ticket. If we were able to implement this today anyway, it’d probably have to be limited to only allow renaming collections within the same database if active index builds are present on the source collection. This is because index builds are pinned to a collection UUID and renaming collections across databases creates a new collection in the target database with a new collection UUID. I was able to make the changes to allow renaming collections with active index builds but eventually ran into the problem where the active index builds MultiIndexBlock PlanExecutor was unusable. I observed the following:
| ||||||||
| Comment by Dianna Hohensee (Inactive) [ 05/Dec/19 ] | ||||||||
|
A renameCollection cmd with a target that already exists will fail unless the dropTarget is set to true. Therefore, renameCollection with dropTarget set to false should never run into a BackgroundOperationInProgress error. We will continue to retry renameCollection if a BackgroundOperationInProgress is encountered in a different form of the cmd. I was going to try to JS test that rename across databases, etc., failed with BackgroundOperationInProgress, but I think I'll try to unit test it instead. | ||||||||
| Comment by Dianna Hohensee (Inactive) [ 05/Dec/19 ] | ||||||||
|
This task poses the question of how to handle renameCollection in the testing infrastructure. Right now, implicitly_retry_background_op_in_progress.js will endlessly retry renameCollection if a background operation in progress error code is returned. This patch will make it so that renameCollection within a database, where the target collection does not exist, should never run into a background op in prog error. However, renameCollection across databases, or when the target collection exists, can still run into background op in progress errors. The first because we create a new Collection object for rename across databases. The second because a target collection that already exists can have an index build in progress that prevents dropping it. | ||||||||
| Comment by Dianna Hohensee (Inactive) [ 06/Aug/19 ] | ||||||||
|
gregory.wlodarek there's a caveat to this ticket: we are only allowing rename during an index build if the rename is within the same database, and there are no index builds in progress on the target collection namespace. | ||||||||
| Comment by Gregory Wlodarek [ 05/Aug/19 ] | ||||||||
|
dianna.hohensee a BF I diagnosed will be fixed with the work being done here that deadlocks during renameCollection(). The check that asserts no index builds are in progress during rename will go away with the work being done here, fixing the deadlock. |