[SERVER-79661] Make "internalRenameIfOptionsAndIndexesMatch" command sharding-aware Created: 03/Aug/23 Updated: 29/Oct/23 Resolved: 09/Oct/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 7.2.0-rc0 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | David Storch | Assignee: | Jordi Serra Torrens |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | pm3229-m1 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||
| Assigned Teams: |
Query Execution
|
||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||
| Sprint: | QE 2023-09-04, Sharding EMEA 2023-10-02, Sharding EMEA 2023-10-16 | ||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||
| Linked BF Score: | 135 | ||||||||||||||||||||||||||||||||
| Description |
|
The "internalRenameIfOptionsAndIndexesMatch" is an internal command designed specifically to be used by $out. It renames a collection but fails if the indexes or options of the destination collection which is being overwritten by the rename do not match those of the source collection. $out works by writing the data to a temporary collection and then as a final step uses "internalRenameIfOptionsAndIndexesMatch" to replace the existing collection with the new data. For $out in sharded clusters, the node executing the $out always targets this command to the primary node of the db primary shard. My understanding is that this targeting behavior is acceptable because the db primary is supposed to coordinate DDL commands. However, the command is not sharding-aware in that it looks like it always performs the rename locally. If either the source or destination collections are unsplittable collections placed on a different shard, then the "internalRenameIfOptionsAndIndexesMatch" becomes a distributed operation. This situation will be possible once we allow unsharded collections to move rather than requiring them to reside on the primary shard. |
| Comments |
| Comment by Githook User [ 09/Oct/23 ] |
|
Author: {'name': 'Jordi Serra Torrens', 'email': 'jordi.serra-torrens@mongodb.com', 'username': 'jordist'}Message: |
| Comment by Jordi Serra Torrens [ 07/Aug/23 ] |
|
Some ideas: Either way, RenameCollectionCoordinator needs to be extended to support the "option and indexes match" functionality. Currently `internalRenameIfOptionsAndIndexesMatch` uses the database exclusive lock to ensure stability between checking the indexes/options and executing the (local) rename. On a sharded cluster, when the collection is not located on the primary shard, the DB lock does no longer provide the required guarantees. Instead, the DDL lock does (for the most part).
Unless createIndexes is changed to either take the DDL lock or become a ShardingDDLCoordinator, the "index match" check will be racy with concurrent index creation. |
| Comment by David Storch [ 03/Aug/23 ] |
|
CC jordi.serra-torrens@mongodb.com ivan.fefer@mongodb.com. Hopefully my description of this task is accurate – let me know if there's anything you'd like to add. |