Uploaded image for project: 'Rust Driver'
  1. Rust Driver
  2. RUST-531

Feature request - Expose the `bulkWrite` command

    • Type: Icon: New Feature New Feature
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None

      First of all, thank you for your awesome work on this driver, and I appologize if this isn't the proper channel to submit feature requests. I couldn't find information on the procedure to submit a feature request and figured the Jira board was probably my best bet.

      The previous rust driver prototype exposed the bulkWrite feature.
      I noticed that the bulkWrite is used internally for things like `Collection::update_many()`, but it is currently not exposed by the driver.

      The way the feature was implemented in the previous driver was pretty flexible and allowed  to reduce the number of roundtrip to mongo required in certain cases by bundling Read/Update/Delete operations together.

      As a more concrete use case, to point out in which situation this can be helpful:

      In a project I work on, we have a migration system built on top of the old driver and for transformations that we can't simply do in a pipeline due to some business logic that is not doable through aggregate pipeline, we iterate through a cursor (with a projection of only the data we need to provide the update) and bundle many UpdateOne operations in a batch; it is much more lightweight to send 1000 batches of 1000 small Update operations which $set a single field than to make 1 000 000 different requests to mongo.

      Since it cannot be done in an `update_many`, the only other method I see that could work with the existing driver is retrieving everything through the cursor instead of just a few field, and then do an `insert_many` call with the option `upsert: true`. This would be even worse than the first method I believe, as at least with the first one we can spawn the single request and join them all at the end so they don't block the cursor iteration, and will end up with significantly less data transiting over the wire and significantly less I/O on the MongoDB as well.

            Assignee:
            Unassigned Unassigned
            Reporter:
            sbergerondrouin@devolutions.net Samuel Bergeron-Drouin
            Votes:
            3 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated: