[DRIVERS-2582] Auto-batching of write operations Created: 27/Mar/23 Updated: 29/Dec/23 Resolved: 21/Dec/23 |
|
| Status: | Closed |
| Project: | Drivers |
| Component/s: | Bulk API, Performance |
| Fix Version/s: | None |
| Type: | New Feature | Priority: | Minor - P4 |
| Reporter: | Alex Bevilacqua | Assignee: | Unassigned |
| Resolution: | Won't Fix | Votes: | 2 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Driver Changes: | Needed | ||||||||
| Quarter: | FY24Q3 | ||||||||
| Description |
|
To improve write throughput from an application that is performing large volumes of insertOne, updateOne and deleteOne operations, the drivers should consider batching these and issuing bulk operations instead. This feature would be disabled by default and would have to temporarily queue operations. Note that queues would need to be maintained for permutations of command options (ex: writeConcern, bypassDocumentValidation, comment on inserts - let, writeConcern and comment for updates/deletes) |
| Comments |
| Comment by Alex Bevilacqua [ 21/Dec/23 ] |
|
After much consideration, this is being closed out as Won't Fix as it introduces a great deal of complexity and opens up a number of edge-cases that could result in negative outcomes. The drivers all support the Bulk Write specification, so this functionality is available. An updated API will also be implemented with DRIVERS-716 which will further improve bulk operation performance. Though auto-batching could improve performance in scenarios where batching wasn't used explicitly and a high throughput workload is being executed - this functionality is better to be added by developers as needed. |
| Comment by Eugene Kang [ 12/Jun/23 ] |
| Comment by Eugene Kang [ 06/Jun/23 ] |
|
It would be nice to have a polling time in ms for this. so we can tune how often the batches go out based on the clients tolerance for latency. |