[SERVER-78253] Allow folks with hashed shard keys to drop the hashed index Created: 20/Jun/23 Updated: 29/Oct/23 Resolved: 02/Aug/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 7.1.0-rc0, 7.0.2, 5.0.22, 6.0.11 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Garaudy Etienne | Assignee: | Ali Mir |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | sharding-nyc-subteam3 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||
| Assigned Teams: |
Sharding NYC
|
||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||
| Backport Requested: |
v7.0, v6.0, v5.0
|
||||||||||||||||||||||||||||
| Sprint: | Sharding NYC 2023-07-10, Sharding NYC 2023-07-24, Sharding NYC 2023-08-07 | ||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||
| Linked BF Score: | 160 | ||||||||||||||||||||||||||||
| Story Points: | 2 | ||||||||||||||||||||||||||||
| Description |
|
The hashed index is only needed to support chunk migrations. However, it hurts write performance. Most collections with hashed shard keys have even data distribution and growth, thus not requiring chunk migrations. As such, we should allow customers to drop the hashed index for hashed shard keys. |
| Comments |
| Comment by Githook User [ 06/Oct/23 ] |
|
Author: {'name': 'Sanika Phanse', 'email': 'sanika.phanse@mongodb.com', 'username': 'sphanse99'}Message: |
| Comment by Githook User [ 26/Sep/23 ] |
|
Author: {'name': 'Sanika Phanse', 'email': 'sanika.phanse@mongodb.com', 'username': 'sphanse99'}Message: |
| Comment by Githook User [ 18/Sep/23 ] |
|
Author: {'name': 'Sanika Phanse', 'email': 'sanika.phanse@mongodb.com', 'username': 'sphanse99'}Message: |
| Comment by Githook User [ 22/Aug/23 ] |
|
Author: {'name': 'Ben Shteinfeld', 'email': 'ben.shteinfeld@mongodb.com', 'username': 'bshteinfeld'}Message: When a shard key contains a hashed component, the ShardFilterer expects This patch also updates the CQF fallback mechanism to enable CQF for |
| Comment by Githook User [ 15/Aug/23 ] |
|
Author: {'name': 'Ben Shteinfeld', 'email': 'ben.shteinfeld@mongodb.com', 'username': 'bshteinfeld'}Message:
|
| Comment by Githook User [ 03/Aug/23 ] |
|
Author: {'name': 'Ali Mir', 'email': 'ali.mir@mongodb.com', 'username': 'ali-mir'}Message: |
| Comment by Githook User [ 02/Aug/23 ] |
|
Author: {'name': 'Ali Mir', 'email': 'ali.mir@mongodb.com', 'username': 'ali-mir'}Message: |
| Comment by Githook User [ 25/Jul/23 ] |
|
Author: {'name': 'Sviatlana Zuiko', 'email': 'sviatlana.zuiko@mongodb.com', 'username': 'szuiko'}Message: Revert " This reverts commit e3d2bc5d42088a9752ac37a3076f3b4e0b98d05a. |
| Comment by Githook User [ 24/Jul/23 ] |
|
Author: {'name': 'Ali Mir', 'email': 'ali.mir@mongodb.com', 'username': 'ali-mir'}Message: |
| Comment by Garaudy Etienne [ 28/Jun/23 ] |
|
For now let's keep the hashed index and just tell people to keep dropping the hashed index. I want us to think through how to get resharding to drop it a little longer... |
| Comment by Max Hirschhorn [ 28/Jun/23 ] |
randolph@mongodb.com for what purpose other than the client application's own reads and writes does this index need to be available? Edit: I think I understand what you're getting at now. If we drop the hashed index without having a non-hashed index then point queries which could have previously used the hashed index will now collection scan if there isn't a non-hashed index available. |
| Comment by Garaudy Etienne [ 28/Jun/23 ] |
|
yes we still need a non-hashed index for everyone with a ranged shard key.
shardCollection and refine don't automatically create the index so they're fine. but yes resharding should stop creating the hash index. |
| Comment by Randolph Tan [ 28/Jun/23 ] |
|
We still need a "non-hashed" shard key index to be available, right? Should we also change/add option for shardCollection/refine/reshard to not create the hash index? |