[SERVER-58413] Create/Find a workload that does CRUD ops while moving chunks around Created: 12/Jul/21 Updated: 29/Jul/21 Resolved: 29/Jul/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Sergi Mateo Bellido | Assignee: | Antonio Fuschetto |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Sprint: | Sharding 2021-07-12, Sharding EMEA 2021-07-26, Sharding EMEA 2021-08-09 | ||||||||||||
| Participants: | |||||||||||||
| Description |
|
This workload should try to take advantage of the Fine-grained Collection Critical Sections that would be implemented as part of PM-2098. This means that the CRUD operations should target a set of chunks that doesn't include chunks being migrated. There are some chunk-related workloads on the workloads repo. This one looks promising. |
| Comments |
| Comment by Antonio Fuschetto [ 26/Jul/21 ] |
|
The previously identified test was rewritten to run very quickly on personal workstations and without the heavy infrastructure required by the workload framework. It has been also changed in order to:
The implementation of the test is available at https://github.com/afuschetto/mongo-scripts/blob/main/performance/move_chunk_with_load.js (read permissions are required). |
| Comment by Antonio Fuschetto [ 14/Jul/21 ] |
|
As initially assumed, a very good candidate to test the performance of the migration protocol and concurrent CRUD operations is the move_chunk_with_load.js in the workloads repository. This test measures the throughput of multiple CRUD operations while a chunk continuously migrates from one shard to another, thereby highlighting the performance penalties introduced by the critical section of the migration protocol and measuring the overall impact on ordinary business operations. Some details on the test implementation follow:
The logic seems more than sufficient for the purposes of this activity, however we could refine measurements and final reports by working on |