[SERVER-55667] Test resharding with real world parameters Created: 31/Mar/21  Updated: 29/Jun/21  Resolved: 29/Jun/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: Lamont Nelson Assignee: Amirsaman Memaripour
Resolution: Done Votes: 0
Labels: PM-234-T-autocommits
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Sprint: Sharding 2021-05-03, Sharding 2021-05-17, Sharding 2021-06-14, Sharding 2021-05-31
Participants:

 Description   

The task is to create a performance test that uses a realistic scenario to measure how resharding impacts customer's workloads.

Rough sketch:
0. Insert initial dataset into a sharded cluster (initial size TBD).
1. Start workload that saturates a cluster's mongod primaries with CRUD operations (ratios TBD)
2. Start a resharding operation on this collection
3. Measure the latency and throughput impact of resharding on the reads/writes
4. Measure the amount of time the resharding coordinators spend in each state.
5. Confirm that the final data in the resharded collection has correct values.

This could be incrementally improved over time. For example, read and write operations could vary with regards to using transactions, the contention to a particular set of shards/documents, the number of shards targeted in a transaction, with concurrent index builds running in the cluster, etc.



 Comments   
Comment by Lamont Nelson [ 31/Mar/21 ]

garaudy.etienne Pulling information about real world usage with regards to hardware resources, data size, and operation type distribution would be useful. Maybe a couple scenarios to start?

Generated at Thu Feb 08 05:37:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.