[DRIVERS-2652] move shard key range logic into the driver vs mongos Created: 12/Jun/23  Updated: 13/Jun/23  Resolved: 13/Jun/23

Status: Closed
Project: Drivers
Component/s: Performance
Fix Version/s: None

Type: Task Priority: Unknown
Reporter: Eugene Kang Assignee: Unassigned
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Driver Changes: Needed

 Description   

Summary

99.99% of customers lean towards hash based sharding to maintain balanced shards. hash based sharding is slow and hurts the cpu for the shard nodes. this affects the overall performance and impacts any poc's we run in the field.

I believe the drivers are constantly polling the cluster and know exactly how many shards are available. If the drivers know how many shards there are, can we just have the drivers produce a random int on the number of shards and use that as the way to distribute the data evenly across the shards without the need for hashing the shardkey?

There are obviously edge cases where a customer may go from x to x+1 shards and we end up with an imbalance of data. Perhaps we can add a weighting factor to the randomint where the x+1 value occurs more than x.



 Comments   
Comment by Bernie Hackett [ 12/Jun/23 ]

In general, we want to make our drivers simpler and easier to reason about. That means, in part, pushing more of the distributed systems functionality into the server, not putting more into the driver.

Comment by Eugene Kang [ 12/Jun/23 ]

xiaochen.wu@mongodb.com rachelle.palmer@mongodb.com 

Generated at Thu Feb 08 08:26:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.