[SERVER-78137] HPDM fetcher/storage transaction batching for indexes Created: 15/Jun/23 Updated: 14/Jul/23 Resolved: 14/Jul/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Huayu Ouyang | Assignee: | Matt Kneiser |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Sprint: | Execution NAMR Team 2023-07-10, Execution NAMR Team 2023-07-24 |
| Participants: |
| Description |
|
2- Investigate to increase the batch size of the storage transaction.
|
| Comments |
| Comment by Matt Kneiser [ 14/Jul/23 ] |
|
I observed decreased throughput by aligning fetcher batches with commit batches. |
| Comment by Matt Kneiser [ 07/Jul/23 ] |
|
I'm seeing quite a bit of batching that we are relying on in onBatchedWriteCommit. For 75 fetcher batches, there are 375 sub-batches. This is an average of 4.9 sub-batches per batch. |