[SERVER-74267] Run YCSB with catalog shard mode Created: 22/Feb/23  Updated: 07/Jun/23  Resolved: 07/Jun/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Kshitij Gupta Assignee: Wenqin Ye
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Sharding NYC
Sprint: Sharding NYC 2023-04-17, Sharding NYC 2023-05-01, Sharding NYC 2023-05-15, Sharding NYC 2023-05-29, Sharding NYC 2023-06-12
Participants:

 Comments   
Comment by Wenqin Ye [ 07/Jun/23 ]

The YCSB workload we used to do this performance test starts with 56 million documents (around 1kb each) and runs YCSB (with 20 million operations) 4 separate times with the following read/write percentages:

  • 95% read, 5% write
  • 50% read, 50% write
  • 100% read
  • 100% update

The averaged results from three runs of the YCSB workload with config shard were compared to the averaged results from three runs of the YCSB workload without config shard. Also note that when when running YCSB with config shard, we improved the hardware of the config server to match that of a regular shard server.

The results can be found in this spreadsheet: https://docs.google.com/spreadsheets/d/1CmtkEm9AQ_BZMFDmUfM0gB1FdihLgp_2Trj1XXFk6L8/edit#gid=1698181027

Overall, there were no statistically significant performance regressions. All performance regressions were within one or two standard deviations of 0% performance regressions. This suggests that the “performance regressions” observed are likely due to random noise inherent to performance tests and not actual performance differences between YCSB with config shard and YCSB without config shard.

Generated at Thu Feb 08 06:26:58 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.