[DRIVERS-2582] Auto-batching of write operations Created: 27/Mar/23  Updated: 29/Dec/23  Resolved: 21/Dec/23

Status: Closed
Project: Drivers
Component/s: Bulk API, Performance
Fix Version/s: None

Type: New Feature Priority: Minor - P4
Reporter: Alex Bevilacqua Assignee: Unassigned
Resolution: Won't Fix Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to DRIVERS-716 Improved Bulk Write API Designing
Driver Changes: Needed
Quarter: FY24Q3

 Description   

To improve write throughput from an application that is performing large volumes of insertOne, updateOne and deleteOne operations, the drivers should consider batching these and issuing bulk operations instead.

This feature would be disabled by default and would have to temporarily queue operations. Note that queues would need to be maintained for permutations of command options (ex: writeConcern, bypassDocumentValidation, comment on inserts - let, writeConcern and comment for updates/deletes)



 Comments   
Comment by Alex Bevilacqua [ 21/Dec/23 ]

After much consideration, this is being closed out as Won't Fix as it introduces a great deal of complexity and opens up a number of edge-cases that could result in negative outcomes. The drivers all support the Bulk Write specification, so this functionality is available.

An updated API will also be implemented with DRIVERS-716 which will further improve bulk operation performance.

Though auto-batching could improve performance in scenarios where batching wasn't used explicitly and a high throughput workload is being executed - this functionality is better to be added by developers as needed.

Comment by Eugene Kang [ 12/Jun/23 ]

xiaochen.wu@mongodb.com rachelle.palmer@mongodb.com 

Comment by Eugene Kang [ 06/Jun/23 ]

It would be nice to have a polling time in ms for this. so we can tune how often the batches go out based on the clients tolerance for latency.

Generated at Thu Feb 08 08:25:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.